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ABSTRACT 


A  significant  number  of  people  worldwide  live  in  rural 
villages  with  only  part-time  electrical  power  or  no  power  at 
all.  In  an  effort  to  address  this  deficiency,  the  U.S. 
National  Renewable  Energy  Laboratory  (NREL)  is  developing 
hybrid  renewable  power  systems  composed  of  photovoltaic 
panels,  wind  turbines,  battery  banks,  and  diesel  generators 
to  deliver  24-hour  power. 

In  keeping  life  cycle  costs  of  these  systems  to  a 
minimum,  an  appropriate  dispatch  strategy  can  contribute  as 
much  to  reducing  cost  as  the  proper  choice  of  system 
architecture.  Current  dispatch  strategies  consider  only 
current  net  load  and  current  state-of-charge  of  batteries. 
The  framework  developed  here  considers  future  net  load 
realizations  in  determining  optimal  dispatch  strategies. 

The  model  is  a  Markov  Decision  Process  solved  using  a 
Policy  Iteration  algorithm.  The  algorithm  produces  a  policy 
that  maps  each  state  that  the  hybrid  renewable  power  system 
can  occupy  to  an  optimal  action.  The  dispatch  policies 
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consider  immediate  and  long  run  ramifications  of  an  action  on 
life-cycle-cost . 

The  results  show  that  over  different  values  of  diesel 
fuel  cost,  battery  wear  cost,  and  loss  of  service  penalties 
the  optimal  strategies  minimize  the  long-run  average  cost. 
Furthermore,  minimal  implementation  costs  make  the  optimal 
strategies  an  attractive  alternative  to  heuristic  strategies 
that  are  currently  used  to  dispatch  hybrid  renewable  power 
systems . 


TABLE  OF  CONTENTS 


Page 

ABSTRACT  .  . . . . . iii 

LIST  OF  FIGURES  .  .  .  . . .  vii 

LIST  OF  TABLES  .....................  x 

ACKNOWLEDGMENTS.  .  .  .  .  . . .  xi 

Chapter 

1.  INTRODUCTION.  ...  .  ..........  1 

1.1  Motivation . 1 

1.2  Generic  Hybrid  Renewable  Power  Systems.  ....  2 

1.3  Problem  Statement  .  .  .  .  .  .  .  .  ...  .  .  .  .  5 

1.4  Objectives.  . . .  .  .  .  8 

2.  REVIEW  OF  RELEVANT  MODELS  . .  11 

2.1  Other  HRPS  Models  . . .  11 

2.2  Model  Presented  Here . .  .  .  .  .  .  .  22 

3.  MODELING  THE  HYBRID  RENEWABLE  POWER  SYSTEM.  ....  27 

3.1  Dynamic  Programming  .  .  . . 27 

3.2  Markov  Decision  Processes  ...........  30 

3.3  Model  Specifics . . . 35 

3.3.1  Stages . 35 

3.3.2  State-Space . . . 36 

3.3.3  Action  Set . .  40 

3.3.4  Cost  Structure . .  41 

3.3.5  Transition  Probabilities.  .  . . 44 


v 


Chapter  Page 

3.4  Solution  of  the  Model  using  Policy 

Iteration . 48 

3.4.1  Policy  Evaluation . 49 

3.4.2  Policy  Improvement . .  .  .  ..  51 

3.5  Policy  Iteration  Algorithm  ..........  52 

3.6  Solution  Procedures.  .  . . 57 

3.7  Evaluation  of  Optimal  Strategies  .  .  .  .  .  .  .  61 

4.  RESULTS  AND  ANALYSIS  ....  .  ......  69 

4.1  Simulation  Results  .  .  . . 69 

4.1.1  Analysis:  Net  Load  Profile  1.  ....  .  .  71 

4.1.2  Analysis:  Net  Load  Profile  2.  .....  .  77 

4.2  HOMER  Results.  ...  .  ........  82 

4.3  Implementation  of  the  General 

Optimization  Framework  .  .  .  . . .  .  86 

4.4  Conclusions.  ......  .  .......  93 

5.  ISSUES  FOR  FURTHER  ANALYSIS.  ......  .  .  .  .  .  .  97 

REFERENCES  CITED.  100 

Appendix 

A.  DESCRIPTION  OF  THE  HYBRID  OPTIMIZATION  MODEL  FOR 

ELECTRIC  RENEWABLES  (HOMER)  .  .  . . 103 

B.  CODE  FOR  POLICY  ITERATION  ALGORITHM  ........  107 

C.  OPTIMAL  DISPATCH  CHARTS  .  .  .  .  .....  .  .  .  .  .126 

vi 


LIST  OF  FIGURES 

Figure  Page 

1.  Generic  Hybrid  Renewable  Power  System  ...  3 

2.  Comparison  of  HRPS  Models  .  .  .  .  .  .  .  .  .  26 

3.  State-Space.  .  . . .  37 

4.  Policy  Iteration  Algorithm  (using 

iterative  policy  evaluation)  for  V"  .  .  .  .  53 

5.  Net  Load  Profile  1  . . 57 

6.  Net  Load  Profile  2 . .  59 

7.  Optimal  Dispatch  Decision  Process  .  .  .  .  .  66 

8.  HOMER  Decision  Process . . 67 

9.  Optimal  Actions,  Net  Load  Profile  1, 

Scenario  1 .  .  .  .  .  .  .  .  .  .  .  . . 69 

10.  Optimal  Action  Profile,  Net  Load 

Profile  1,  Scenario  1  ..........  .  70 

11.  Comparison  of  Dispatch  Strategies,  72 

Net  Load  Profile  1.  .  .  .  . . 

12.  Summary  of  Dispatch  Strategy  Results,  73 

Net  Load  Profile  1.  .  .  .....  .  .  .  .  . 

13.  Quality  of  Service  Analysis,  Net  Load 

Profile  1 ................  .  76 

14.  Comparison  of  Dispatch  Strategies,  Net 

Load  Profile  2 . .  .  .  ...  .  .  78 

vii 


Figure  Page 

15.  Summary  of  Dispatch  Strategy  Results,  Net 

Load  Profile  2  .  .  .  .  .  .  .  .  .  .  .  .  .  .  79 

16.  Quality  of  Service  Analysis,  Net  Load 

Profile  2 .  . . . .  .  81 

17.  Optimal  Strategy,  Net  Load  Profile  1, 

Scenario  1  .  ........  127 

18.  Optimal  Strategy,  Net  Load  Profile  1, 

Scenario  2  .  .  . . . .  .  128 

19.  Optimal  Strategy,  Net  Load  Profile  1, 

Scenario  3  ......  .  .'  .  .  .  .  .  ...  129 

20.  Optimal  Strategy,  Net  Load  Profile  1, 

Scenario  4  .  .  . . .  .  ...  .130 

21.  Optimal  Strategy,  Net  Load  Profile  1, 

Scenario  5  .  .  .  .  .  .  ...  .  .  .  .  .  .  131 

22.  Optimal  Strategy,  Net  Load  Profile  1, 

Scenario  6  .  . . .  .  .  .  .  .  .  .  132 

23.  Optimal  Strategy,  Net  Load  Profile  1, 

Scenario  7  .  .  .  .  .  .  .  .  .  .  . . 133 

24.  Optimal  Strategy,  Net  Load  Profile  1, 

Scenario  8  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  134 

25.  Optimal  Strategy,  Net  Load  Profile  1, 

Scenario  9  .  .  . . .  .  .  .  .  .  .  135 

26.  Optimal  Strategy,  Net  Load  Profile  2, 

Scenario  1  .  .  . . .  .  .  .  .  .  .  136 

27.  Optimal  Strategy,  Net  Load  Profile  2, 

Scenario  2  .  .  .  . . .  ....  .  .  137 


viii 


Figure  Page 

28.  Optimal  Strategy,  Net  Load  Profile  2, 

Scenario  3  .  .  .  . . 138 

29.  Optimal  Strategy,  Net  Load  Profile  2, 

Scenario  4  .  .....  139 

30.  Optimal  Strategy,  Net  Load  Profile  2, 

Scenario  5  ...............  .  140 

31.  Optimal  Strategy,  Net  Load  Profile  2, 

Scenario  6 . 141 

32.  Optimal  Strategy,  Net  Load  Profile  2, 

Scenario  7  .  .  .  . . .  ....  142 

33.  Optimal  Strategy,  Net  Load  Profile  2, 

Scenario  8 . 143 

34.  Optimal  Strategy,  Net  Load  Profile  2, 

Scenario  9  .  ...........  144 


IX 


LIST  OF  TABLES 

Table  Page 

1.  Explanation  of  Terms  (Kandil)  .......  14 

2.  Explanation  of  Terms  (Barley)  .......  18 

3.  Explanation  of  Terms  (Bakirtzis) .  .  .  .  .  .  21 

4.  Typical  Characteristics  of  a  Dynamic 

Program  . .  .....  28 

5.  Scenarios . 60 

6.  Results  from  HOMER.  Net  Load  Profile  1  .  .  84 

7.  Results  from  HOMER.  Net  Load  Profile  2  .  .  85 

8.  Optimal  Strategies  vs.  Next  Best 

Heuristic  .  . . 90 

9.  Benefit-Cost  Analysis,  Net  Load 

Profile  1  .  .  .  ...  .  .  .  .  ...  .  .  .  .  92 

10.  Benefit-Cost  Analysis,  Net  Load 

Profile  2 . 93 


x 


ACKNOWLEDGMENTS 


First  and  foremost,  I  wish  to  express  my  deepest 
gratitude  to  my  friend  and  mentor  Jeff  Grobman  who  in  more 
than  one  instance  kept  me  from  wandering  aimlessly  down  the 
wrong  path.  I  hope  some  day  to  repay  him  for  the  accurate 
and  timely  advice  he  provided  me  over  the  past  two  years. 

Many  thanks  to  Dr.  R.E.D.  Woolsey  whose  unique  outlook 
on  life  and  the  Operations  Research  profession  can  be  a 
lesson  to  us  all  about  the  power  of  a  vision  and  the 
perseverance  to  carry  it  out. 

I  also  wish  to  thank  the  following  members  of  my 
committee:  Dr.  Robert  Frost,  Dr.  William  Navidi,  and  Dr. 
Roderick  Eggert. 

I  would  also  like  to  acknowledge  the  help  of  Dr.  Peter 
Lilienthal  and  Mr.  Tom  Lambert  of  the  National  Renewable 
Energy  Laboratory.  Peter  planted  the  idea  that  got  me 
started  on  this  research  effort  and  Tom  provided  me  with 
encouragement  and  timely  assistance  with  HOMER. 


xi 


Of  course,  I  cannot  forget  the  support  provided  by  my 
wife,  Karen,  who  endured  many  lonely  days  and  nights  over  the 
past  few  years.  And  last  but  not  least,  my  son  Will  whose 
carefree  attitude  (what  other  kind  does  a  two-year-old  have?) 
kept  me  focused  on  what's  really  important  in  life. 


1 


Chapter  1 
INTRODUCTION 


1.1  Motivation 

Worldwide,  some  two  billion  people  live  without 
electricity  (Lilienthal  1995,  1) .  These  people  who  lack 
such  things  as  clean  water,  refrigerated  foods  and 
medicines,  telephones,  and  radios,  miss  the  social, 
economic,  and  health  benefits  that  these  necessities  afford 
the  rest  of  the  world.  The  goal  of  the  U.S.  National 
Renewable  Energy  Laboratory  (NREL)  is  to  integrate 
affordable  and  practical  renewable  energy  technologies,  such 
as  photovoltaics  (PV)  and  wind  power,  into  rural  economic 
development.  By  doing  so,  these  renewable  energy  systems 
could  provide  many  benefits,  such  as  water  pumping 
capability,  electrification  of  schools,  community  centers 
and  health  clinics,  and  desalinization,  or  disinfection  of 
water. 

In  addition  to  those  people  who  do  not  have 
electricity,  a  sizable  number  of  people  worldwide  live  in 
rural  villages  with  only  part-time  electrical  service 
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provided  through  diesel  generation.  The  Renewables  for 
Sustainable  Village  Power  group  at  NREL  is  developing  hybrid 
renewable  power  systems  (HRPS)  that  include  small  wind 
turbines,  photovoltaic  panels,  battery  storage,  and  diesel 
generators  to  provide  24-hour  power  to  remote  villages. 

These  hybrid  systems  often  outperform  the  generator-only 
systems  with  higher  quality,  greater  reliability,  lower 
cost,  and  reduced  environmental  degradation  (Flowers  1997) . 

1.2  Generic  Hybrid  Renewable  Power  Systems 

A  generic  HRPS  is  shown  in  figure  1.  The  "AC 
generators"  represent  various  sources  that  produce 
alternating  current.  Such  sources  include  AC  wind  turbines 
and  AC  diesel  generators.  it  is  the  diesel  generator  that 
we  are  trying  to  incorporate  into,  or  build  a  HRPS  around, 
to  provide  24-hour  power.  The  "DC  generators"  represent 
various  energy  sources  that  produce  direct  current.  Such 
devices  include  DC  wind  turbines,  DC  diesel  generators  and 
photovoltaic  panels.  "AC  loads,"  represent  typical  fixtures 
or  appliances  that  require  alternating  current  similar  to 
those  that  one  might  find  in  any  U.S.  household.  "DC 
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Figure  1.  Generic  Hybrid  Renewable  Power  System 
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Loads,"  represent  special  fixtures  or  appliances  such  as 
lights,  refrigerators,  or  water  pumps  that  require  direct 
current.  Direct  current  appliances  are  not  common  in  the 
United  States  but  are  often  used  with  renewable  energy 
systems.  Almost  all  HRPS  have  an  inverter,  a  rectifier,  or 
both.  The  inverter,  through  a  special  configuration  of 
diodes  and  other  electronic  components,  converts  direct 
current  into  alternating  current  for  appropriate  end-use  by 
the  consumer.  The  rectifier  does  the  opposite  and  converts 
alternating  current  into  direct  current  for  use  by  DC  loads 
or  for  storage  in  the  "DC  storage"  component,  namely 
batteries. 

Control  of  the  energy  flows  into  and  among  the 
components  of  this  system  can  be  very  complicated.  For 
example,  in  any  typical  hour,  one  would  have  to  check  the 
level  of  the  AC  and  DC  loads  (electrical  demand).  If  the 
power  provided  by  AC  generators  (wind  turbines)  combined 
with  the  inverted  DC  energy  stored  in  the  batteries  and  the 
inverted  DC  generator  power  is  high  enough,  then  the  diesel 
generator  could  be  left  off.  If  the  AC  and  DC  generator 
power  is  higher  than  the  loads,  then  one  might  want  to  store 
some  of  this  energy  for  future  use.  This  would  require  the 
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AC  energy  be  rectified  and  stored  in  the  batteries,  while 
the  DC  energy  could  be  stored  directly.  If  the  AC  and  DC 
loads  happen  to  exceed  AC  and  DC  generation  and  battery 
state-of-charge,  then  the  diesel  engine  must  be  turned  on  to 
meet  the  net  load.  In  this  case  the  challenge  is  to 
determine  the  power  level  the  generator  should  be  set  at  in 
order  to  meet,  most  economically,  the  net  load,  taking  into 
consideration  future  net  loads. 

1 . 3  Problem  Statement 

One  barrier  to  using  renewable  technologies  in  rural 
villages  is  the  lack  of  analytical  tools  that  accurately 
compare  both  conventional  and  renewable  energy  supply 
options.  Assessing  the  least-cost  mix  of  supply 
technologies  is  a  difficult  analytical  problem  that  depends 
on  several  factors.  Among  them  are  the  quality  and 
uncertainty  of  the  various  wind  and  sun  resources,  the  costs 
of  acquiring  equipment,  fuel  costs,  and  site  specific 
descriptions  of  the  daily  and  seasonal  variations  in  the 
electrical  demand.  Consequently,  inexperienced  users  who 
want  to  investigate  the  technical  and  financial  performance 
of  hybrid  renewable  power  systems  have  a  difficult  time 
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choosing  the  right  one  for  their  needs.  To  aid  in  choosing 
a  HRPS,  screening  and  analysis  models,  such  as  HYBRID  2 
(Barley  1996,  66)  and  the  Hybrid  Optimization  Model  for 
Electric  Renewables  (Lilienthal  1995),  are  available.  They 
are  useful  for  feasibility  studies  that  provide  data  on 
several  topics  including  comparison  among  existing 
technologies  in  terms  of  economics  and  performance  leading 
to  an  optimal  design  for  any  particular  application.  HYBRID 
2  is  a  detailed  simulation  model  which  analyzes  system 
performance  over  time,  once  an  architecture  and  dispatch 
strategy  have  been  decided  upon.  HYBRID  2  is  very  time 
consuming  to  set  up  and  run,  so  it  is  important  to  screen 
out  sub-optimal  system  designs  avoiding  unnecessary 
simulation  effort.  One  such  screening  model  is  the  Hybrid 
Optimization  Model  for  Electric  Renewables  (HOMER)  developed 
by  Peter  Lilienthal  of  NREL.  HOMER  determines  a  set  of 
optimal  or  near  optimal  system  designs,  including 
appropriate  dispatch  strategies,  for  further  analysis  by 
HYBRID  2.  There  has  been  an  ongoing  effort  to  implement  a 
predictive  dispatch  capability,  such  as  the  one  presented 
here,  into  HOMER.  While  the  effort  here  is  a  general 
framework,  HOMER  is  the  platform  into  which  this  framework 
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will  be  incorporated  at  some  future  time.  A  description  of 
HOMER  is  provided  in  Appendix  A  to  give  the  reader  an  idea 
of  the  similarities  and  differences  between  the  general 
framework  and  HOMER. 

Screening  models  such  as  HOMER  determine  the  lowest 
life-cycle-cost  (LCC)  HRPS  through  analysis  of  architecture 
and  dispatch  strategies.  A  sound  dispatch  strategy,  the 
decision  when,  and  if,  to  turn  the  generator  on,  how  long  it 
runs,  and  whether  or  not  to  use  batteries,  can  contribute  as 
much  to  minimizing  LCC  as  the  selection  of  system  components 
(Lilienthal  1995) .  Given  a  net  load  (village  electrical 
demand  minus  currently  available  renewable  power),  typical 
dispatch  strategies  considered  are  1)  load-following— just 
meeting  the  net  load;  2)  cycle-charging — exceeding  the  net 
load  in  order  to  charge  the  batteries,  or  3)  combined — 
load-following  or  cycle-charging  based  on  a  predetermined 
set  of  rules.  Chapter  3,  section  7  contains  a  complete 
description  of  these  strategies. 

The  problem  with  existing  dispatch  strategies  is  that 
dispatch  decisions  are  not  optimal  in  any  sense.  They  are 
based  on  heuristic  rules  that  employ  minimum/maximum  battery 
state-of-charge  thresholds  or  minimum/maximum  generator  run- 
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time  criteria  to  determine  generator  settings.  Furthermore, 
dispatch  decisions  are  made  during  the  current  time  period 
regardless  of  the  expected  net  load  next  period.  An 
improved  implementation  makes  a  dispatch  decision  now, 
taking  into  account  the  future  net  loads  and  optimizing 
dispatch.  At  a  minimum  dispatch  decisions  based  on  mean 
values  of  future  net  loads  should  help  to  reduce  system 
costs  by  taking  advantage  of  a  "good"  system  status  to 
mitigate  the  effects  of  a  "bad"  system  status  some  time  in 
the  future.  In  practice,  the  future  net  loads  are 
stochastic  and  one  might  only  know  the  mean  and  variance  of 
the  net  loads  in  the  future,  described  by  some  probabilistic 
distributions. 

1.4  Objectives 

The  goal  of  this  research  is  to  develop  a  general 
framework  for  determining  optimal  dispatch  strategies  for 
the  operation  of  hybrid  renewable  power  systems.  The 
general  framework  will  serve  three  purposes.  The  first 
purpose  is  as  a  tool  to  be  incorporated  into  screening 
models  for  determining  optimal  dispatch  strategies  for  a 
specific  hybrid  renewable  power  system.  The  second  purpose 
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is  to  provide  an  operator  a  simple,  understandable  decision 
making  tool  for  determining  how  to  run  a  hybrid  renewable 
power  system,  given  battery  state-of-charge,  electrical 
demand,  renewable  resource,  and  time-of-day  considerations. 
The  third  is  to  provide  a  baseline  for  future  implementation 
of  the  framework  into  a  PC-based  controller. 

The  framework  developed  here  is  unique  in  that  it 
simultaneously  considers  generator  costs,  battery  wear 
costs,  and  loss  of  load  penalties  in  dispatch  decisions 
where  other  models  consider  only  some  subset  of  the  three. 
This  gives  the  customer  the  flexibility  to  trade  service 
availability  for  lower  operating  costs  and  vice  versa,  if  so 
desired. 

With  this  model,  dispatch  decisions  can  be  made 
independently  of  both  the  initial  and  final  conditions  of 
the  system.  This  is  significant  for  two  reasons.  The  first 
reason  is  that  the  system  user  does  not  have  to  make 
potentially  sub-optimal  decisions  about  battery  charge 
levels  or  generator  settings  at  the  beginning  or  end  of  a 
planning  period.  The  second  reason  is  that,  in  the  event  of 
a  system  stoppage,  optimal  dispatch  decisions  can  be  made 
immediately,  regardless  of  the  state  of  the  system  or  the 


10 


experience  of  the  operator.  Other  models  require 
specification  of  initial  and  terminal  system  conditions  or, 
in  the  event  of  system  failure,  significant  operator 
involvement  in  re-establishing  optimal  dispatch  conditions. 

.  This  framework  is  predictive  in  nature.  It  accounts 
for  the  stochastic  nature  of  the  wind  and  sun  resources  and 
electrical  demand  anticipating  potential  net  load 
realizations  in  future  periods.  In  doing  so,  it  considers 
both  the  immediate  and  long-term  effects  of  any  dispatch 
decision.  Other  HRPS  models  are  deterministic  in  nature  or 
only  consider  the  impact  of  net  load  realizations  over  a 
short  planning  horizon. 

The  model  is  also  flexible  enough  to  consider  any  non- 
linearities  in  cost  or  constraint  functions  where  many  other 
models  must  use  piece-wise  linear  approximations  to  non¬ 
linear  functions. 
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Chapter  2 

REVIEW  OF  RELEVANT  MODELS 


2.1  Other  HRPS  Models 

While  the  economic  value  of  forecasting  wind  has  been 
established  in  relation  to  utilities  incorporating  wind 
turbine  systems  into  existing  grid  structures  (Miligan  1995, 
10) ,  little  effort  has  been  given  towards  using  knowledge  of 
expected  net  load  distributions  for  predictive  dispatch  of 
hybrid  renewable  power  systems. 

Due  to  the  difficult  nature  of  solving  such  stochastic 
models,  many  dispatch  models  are  deterministic  in  nature. 

In  fact,  of  the  hybrid  renewable  power  systems  simulation 
software  available  on  the  market,  none  have  a  predictive 
dispatch  capability  (Barley,  1993) . 

One  deterministic  approach  to  dispatch  for  hybrid 
renewable  power  systems  is  presented  in  Lilienthal  (1995) 
where  required  diesel  power  in  each  of  8,760  hours  of 
operation  is  solved  for  simultaneously  using  past,  current, 
and  point  estimates  of  future  net  load  values.  While 
illuminating,  this  approach  does  not  account  for  the 
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stochastic  nature  of  net  loads.  This  linear  programming 
approach  to  dispatch  does  provide  automatic  sensitivity 
analysis  of  important  parameters  in  the  model.  It  also 
serves  as  the  framework  around  which  the  current  version  of 
HOMER  was  built. 

Hancock  (1995)  develops  a  model  of  an  autonomous  remote 
area  power  system  (RAPS)  consisting  of  photovoltaic,  diesel 
generator,  and  battery  storage  components.  His  model 
optimizes  dispatch  strategy  over  a  24-hour  period  using 
discrete  one-hour  decision  intervals.  Hancock  considers 
generator  fuel  cost  using  a  linear  fuel  consumption  curve, 
but  does  not  consider  battery  wear  cost.  He  does  consider 
loss  of  load  (not  meeting  demand)  by  applying  an  incremental 
loss  of  load  penalty  (lip)  : 

lip  =  5  x  (i incremental  fuel  consumption )  x  (cos?  of  fuel)  $  /  kWh 

His  method  determines  a  dispatch  strategy  using  a  mixed 
linear  programming-dynamic  programming  (LPDP)  methodology 
after  Bannister  and  Kaye  (1991)  to  model  the  remote  solar- 
diesel-battery  system.  The  method  uses  a  typical  dynamic 
programming  approach  where  linear  programming  is  used  in 
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each  time  step  to  determine  a  "cost-to-go"  function.  The 
"cost-to-go"  is  a  function  of  the  storage  level  (battery 
charge) ,  which,  in  turn,  determines  an  appropriate  generator 
setting.  The  LPDP  method  is  a  deterministic  approach  that 
assumes  perfect  knowledge  of  future  net  loads.  Hancock's 
model  combines  all  of  the  familiar  features  of  linear 
programming  and  the  flexibility  of  dynamic  programming  to 
determine  optimal  dispatch  decisions  over  the  specified 
planning  horizon. 

Kandil  (1991)  models  a  HRPS  consisting  of  solar,  wind 
and  conventional  diesel  generation,  as  well  as  a  storage 
capability.  He  optimizes  the  dispatch  of  the  autonomous 
system  over  a  period  of  24  hours  using  discrete  one-hour 
time  intervals.  He  considers  generator  cost,  but  not 
battery  wear  cost.  Loss  of  load  is  addressed  by  using  a 
system  constraint  that  requires  demand  must  be  met.  Kandil 
uses  a  minimum  cost  generalized  network  flow  model  to 
minimize : 

TEC  -  £  Z  OMC  x  FLy  +  FUELC  (  2.1) 

i  j 

The  terms  in  equation  2.1  are  defined  in  table  1. 
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Table  1.  Explanation  of  Terms  (Kandil) 


TERM 

MEANING 

TEC 

=  Total  Energy  Cost 

OMC 

=  Operation  and 
Maintenance  cost 

i 

=  z'th  operating  hour 

j 

=  j th  renewable 
component  ( PV 
panel,  wind 
turbine,  etc) 

FLtj 

=  Power  output  of 
renewable  energy 
source 

FUELC 

=  Fuel  cost  of  diesel 

In  order  to  use  a  network  flow  to  model  the  non-linear 
functions,  he  uses  a  piece-wise  linear  approximation  of  non¬ 
linear  cost  function,  specifically  the  diesel  fuel  cost  (FC) 
curve  which  is  expressed  as 


FC  =  aP2  +bP  +  c 
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where  P  is  the  power  setting  of  the  diesel  system  in  any  one 
hour  and  a,  b,  and  c  are  constants.  The  new  TEC  then 
becomes 

TEC  =  YI4OMC*FLi]+Y.hCk 

ij  k 

where  k  is  the  number  of  segments  in  the  piece-wise 
linearization,  is  the  unit  fuel  cost  for  segment  k ,  and 

Pk  is  the  power  output  in  segment  k  .  The  network  is 
deterministic  in  nature  in  that  wind,  sun  and  demand  are 
assumed  to  be  known  with  certainty. 

The  results  of  his  model  show  that  the  generator  is 
never  turned  on  in  the  24-hour  period  over  which  the  system 
is  optimized.  This  is  significant  in  that  generator  fuel 
usage  is  the  largest  variable  cost  of  running  a  HRPS.  The 
result  is  not  unexpected,  however.  The  system  starts  with 
1000  kWh  of  charge  in  the  batteries  and  the  highest  demand 
in  any  one-hour  is  300  kWh.  Plus,  the  wind  and  solar 
resources  exceed  demand  in  12  of  the  24  hours.  So,  the 
chance  that  the  generator  would  have  to  be  used  for  meeting 
excess  demand  was  remote  to  start  with. 
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Barley  (1996)  develops  a  simple,  understandable 
heuristic  to  address  dispatch  strategies.  He  considers  a 
HRPS  composed  of  wind  turbines,  a  diesel  generator  and 
battery  storage.  The  objective  of  his  work  is  to  minimize 

N  N 

Cop  =  cy  (Fi  AO  +  cbwbattcap^  -  soct  ] +  (2.2) 

/= 1  j=l 

The  terms  in  equation  2.2  are  defined  in  table  2. 

Barley  analyzes  actual  data  collected  at  three  HRPS  sites  to 
gain  insight  into  the  level  of  improvement  that  can  be 
gained  by  having  an  "ideal"  predictive  dispatch  strategy. 

In  doing  so.  Barley  analyzes  data  for  an  entire  year  and 
uses  the  data  as  a  "yardstick"  against  which  to  measure  his 
heuristic  dispatch  methods. 

His  heuristic  uses  wind-to-average  load  ratios  (WLR) , 
diesel-to-peak  load  ratios  (DLR)  and  fuel-to-battery  cost 
ratios  (FBCR)  to  determine  an  appropriate  dispatch  strategy, 
specifically  load-following  or  cycle-charging.  The  ratios 
are  defined  as 

Average  Wind  Power 

WLR  = - 

Average  Load  ( Demand ) 
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Diesel  Size 

DLR  = - 

Peak  Load  (Demand) 

Bcf 

FBCR  = - - f—x — —  (2.3) 

cbw  +  Acf( - 1) 

Vrt 

In  equation  2.3,  B  is  the  diesel  fuel  consumption  rate  at  no 

load  (the  "cost"  in  terms  of  fuel  consumption  for  turning 

the  generator  on)  and  A  is  the  incremental  generator  fuel 
consumption  rate. 

Barley  utilizes  a  linear  diesel  fuel  consumption  curve 
and  assumes  the  diesel  generator  is  sized  to  at  least  meet 
peak  electrical  demand.  Therefore,  the  potential  for  loss 
of  load  is  minimized  to  the  greatest  extent  possible  under 
his  framework.  Barley's  is  a  simple  and  effective  approach 
to  dispatch,  but  does  not  formally  show  optimality.  His 
analysis  considers  a  number  of  sites  and  is  based  on 
empirical  data  in  contrast  to  the  more  general  Markov 
Decision  Process  framework  presented  here. 

Contaxis  and  Kobouris  (1991)  address  prediction  of  wind 
and  electrical  demand  for  a  wind/diesel  hybrid  system 
currently  in  use  in  the  Greek  Islands.  The  HRPS  they 
consider  has  several  diesel  generators  and  wind  turbines, 


Table  2.  Explanation  of  Terms  {Barley) 


TERM 

MEANING 

Cop 

= 

Operating  cost  for 
simulation  period 

Cf 

Diesel  fuel  cost 

Fi 

= 

Diesel  generator 
fuel  consumption 
rate 

At 

= 

Time  step,  hours 

cbw 

— 

Cost  of  battery 
wear 

battcap 

= 

Battery  Capacity 

soc 

= 

State  of  charge  of 
the  battery 

i 

= 

Index  of  time  Step 

N 

= 

Total  number  Of 
time  steps  in 
simulation  period 

■  >r 

= 

Indicates  summation 
of  positive  term 
only 
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but  no  storage  capability  (batteries) .  The  system  is 
assumed  to  be  autonomous,  requiring  no  manual  dispatch. 

Unit  commitment ,  or  determining  the  number  of  generators  to 
utilize  over  a  fixed,  finite  planning  horizon  (T ) ,  is  the 
focus  of  their  work.  Their  model  uses  a  two  step  process: 
the  first  step  is  to  forecast  the  electrical  demand  and  wind 
resource  over  the  planning  horizon;  and  the  second  step  is 
to  use  a  heuristic  algorithm  to  determine  unit  commitment  in 
any  given  time  period. 

The  wind  and  demand  forecasts  are  assumed  to  be 
normally  distributed  random  variables  and  are  forecast  using 
ARMA  models.  Contaxis  and  Kobouris  found  forecasting  the 
wind  resource  to  be  a  difficult  undertaking.  Forecasting 
demand,  however,  is  more  reliable  since  demand  can  be 
represented  by  a  stationary  process  with  strong 
regularities . 

Rather  than  using  a  heuristic  algorithm,  Bakirtzis  and 
Gavanidou  (1992)  use  a  more  formal  approach  to  predictive 
dispatch.  They  analyze  the  operation  of  a  HRPS  in  the 
context  of  a  stochastic  dynamic  program.  In  essence,  they 
use  a  very  dense  decision  tree  and  roll  it  back  to  determine 
optimal  dispatch  strategies.  The  system  under  consideration 
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has  multiple  diesel  generators  and  wind  turbines  as  well  as 
PV  and  battery  storage.  The  goal  of  their  analysis  is  to 
determine  diesel  generator  unit  commitment  and  to  provide  a 
scheme  for  optimal  operation  of  the  system.  Specifically, 
their  objective  is  to  minimize  the  expected  value  of  fuel 

T  nd 

consumption,  E{Ff}=  Fi(Pf).  (0]}  r  over  a  24-hour  period. 

t=\  i=l 

The  terms  of  the  objective  function  are  explained  in  table 
3.  They  use  a  linear  fuel  consumption  curve,  but  do  not 
consider  battery  wear  costs.  Loss  of  load  is  a  minor  issue 
in  their  model  since  committing  an  additional  generator  can 
avoid  loss  of  load. 

Bakirtzis  and  Gavanidou  find  their  prediction  of  demand 
to  be  satisfactory,  but  find  wind  and  sun  resources  to  be 
difficult  to  predict.  For  defining  their  state-space  they 
assume  that  the  wind,  sun,  and  demand  distributions  are 
normally  distributed  and  statistically  independent.  This 
facilitates  calculation  of  the  transition  probabilities  from 
one  state  to  the  next,  because  the  transition  probability  is 
simply  the  product  of  the  three  component  probabilities. 
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Table  3.  Explanation  of  Terms  (Bakirtzis) 


TERM 

MEANING 

Fj1  ~ 

Fuel  cost  over 
period  of  length  T 

t  = 

Time  index 

T  = 

Planning  horizon 

i  = 

ith  diesel 
generator 

nd 

Total  number  of 
generators 

Fi  = 

Fuel  used  by  the 
ith  generator 

II 

/ — \ 

of 

Power  setting  of 
the  ith  generator 
in  time  period  t 

They  use  a  discretization  method  for  wind,  sun  and  load 
distributions,  similar  to  the  one  used  here.  The  result  of 
their  modeling  effort  is  a  "look-up"  table  which  has  optimal 
battery  state  of  charge  settings  for  each  hour  of  the  day 
with  diesel  generator  settings  that  are  calculated 
separately. 

Their  assumption  of  a  specific  time  horizon  has  an 
effect  on  the  initial  and  final  states  of  their  system  and 
the  optimal  action  at  time,  T.  Specifically,  Bakirtzis  and 


22 


Gavanidou  assume  that  the  battery  bank  must  be  fully  charged 
to  a  level  of  400  kWh  at  the  beginning  and  end  of  the  24- 
hour  period.  Such  assumptions  of  where  the  system  should 
"start"  and  where  it  should  "end"  in  terms  of  system  status 
could  possibly  lead  to  sub-optimal  solutions.  By  assuming  a 
specific  time  period  over  which  the  system  must  be 
optimized,  it  may  well  be  that  the  "optimal"  dispatch  up  to 
and  including  period  T  may  produce  sub-optimal  conditions 
for  periods  T+ 1  onward.  The  necessity  of  making  system 
status  assumptions  and  time  horizon  specifications  can  be 
avoided  by  assuming  the  HRPS  can  be  modeled  as  an  infinite 
time  horizon,  Markov  Decision  Process  (MDP) . 

2 . 2  Model  Presented  Here 

Assuming  an  infinite  time  horizon  MDP  is  advantageous  • 
because  its  Bellman  equations 

V(s)  =  max[u(s,a)  +  f3^V(s')p(ds'\s,a)\  (2.4) 

have  attractive  properties.  In  particular,  if  we  define  the 
right  hand  side  of  equation  2.4  as  the  Bellman  operator. 
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r(fF)(,s)  =  max[u(s,a)  +  /3^W(s')p(ds'\s,a)] 

equation  2.4  can  now  be  written  as,  V  =  r(V),  which  means  V 
is  a  fixed  point  of  T.  This  implies  that  for  the  discrete 
dynamic  system  model  used  here,  unique  equilibrium  states 
for  each  state-value  V(s )  exist.  Therefore,  starting  from 
any  initial  state,  the  system  will  evolve  towards 
equilibrium  (Rust  1996,  635).  A  direct  result  of  this  is 
that  the  policy  given  by  the  solution  of  the  infinite 
horizon  MDP  constitutes  an  optimal  decision  rule  for  each 
state  the  system  can  occupy  (Rust  1996,  636) .  Thus,  unlike 
heuristic  methods  that  may  only  be  optimal  under  specific 
conditions,  the  algorithm  is  optimal  regardless  of  the 
conditions. 

The  model  developed  here  can  accept  any  combination  of 
linear  or  non-linear  cost  or  constraint  functions  and  can 
evaluate  complex,  non-linear  relationships  and  achieve 
globally  optimal  solutions.  The  distributions  of  the  wind, 
sun,  and  demand  that  make  up  the  net  load  distribution 
(energy  demanded  minus  energy  provided  by  renewable 
resources)  can  be  anything,  meaning  statistical  independence 
of  the  three  components  is  not  required  as  long  as  the  net 
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load  distribution  can  be  discretized.  Furthermore,  the 
framework  anticipates  the  net  load  realizations,  so  it  is 
predictive  in  nature.  The  model  assumes  the  system  can  be 
run  autonomously,  but  the  product  of  model  also  allows  for 
simple,  manual  dispatch  of  the  system.  The  look-up  tables  it 
produces  are  optimal  generator  settings  that  allow  an 
inexperienced  operator  to  run  the  system. 

Generally,  the  HRPS  considered  for  remote  villages  have 
only  one  generator  and  therefore  do  not  have  the  luxury  of 
using  one  more  than  one  unit,  so  unit  commitment  is  not 
considered  although  it  could  easily  be  incorporated  into  the 
model . 

The  framework  is  flexible  in  the  sense  that  loss  of 
load  cost,  the  size  of  which  is  determined  by  the  magnitude 
of  the  loss  of  load  penalty,  can  be  traded  against  generator 
and  battery  wear  costs,  if  desired.  Net  load  profiles  are 
examined  where  loss  of  load  is  likely  to  occur,  giving  a 
realistic  set  of  results  to  examine. 

The  disadvantage  of  using  an  MDP  is  that  it  is 
computationally  expensive  relative  to  a  linear  programming 
or  a  finite  dynamic  programming  approach  to  the  dispatch 
problem.  This  is  due  to  the  "curse  of  dimensionality"  (Rust 
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1996,  625)  encountered  in  defining  too  large  a  state-space 
and/or  action  set.  Also,  by  discretizing  the  state-space 
and  action  set,  the  optimal  solutions  contained  here  are 
limited  to  those  combinations  of  state,  action  and  net  load 
realizations  defined  by  the  state-space.  It  can  be  shown, 
however,  that  fairly  "coarse"  discretizations  do  a  good  job 
of  approximating  underlying  continuous  functions  (Rust  1996, 
659).  Figure  2  shows  a  comparison  of  this  framework  to  the 
other  models  described  here. 
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Chapter  3 

MODELING  THE  HYBRID  RENEWABLE  POWER  SYSTEM 


In  this  chapter  I  analyze  the  hybrid  renewable  power 
system  dispatch  problem  from  a  theoretical  approach  as 
suggested  by  Ethridge  (1995,  133)  and  then  outline  the 
specific  assumptions  made  in  modeling  the  HRPS.  The  problem 
of  operating  a  HRPS  can  be  framed  in  terms  of  a  sequential 
decision  making  process  over  time.  Consequently,  it  is  well 
suited  for  treatment  as  a  dynamic  program,  specifically,  a 
Markov  Decision  Process. 

3.1  Dynamic  Programming 

Dynamic  programming  is  a  useful  framework  that 
practitioners  apply  to  a  variety  of  multi-stage  decision 
problems  occurring  over  time  or  when  choices  are  made 
sequentially.  The  operation  of  a  HRPS  fits  nicely  into  this 
category.  Problems  that  are  formulated  and  solved  using 
dynamic  programming  have  several  common  characteristics. 

They  are  shown  in  table  4. 
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Table  4.  Typical  Characteristics  of  a  Dynamic  Program 


Characteristic 

Notation 

Time  Index  (or  Stages) 

te{0,\X.,T},T<co 

State-Space 

S  such  that  seS,  s'  e  S  ■ 

s  is  a  state  the  system  is 
currently  occupying 

s'  is  a  state  to  which  the 
system  can  transition 

Action  Set 

A  such  that  a  e  A 

a  is  an  action  available  to 
the  decision-maker  in  state  s 

Cost 

C°s.  the  cost  of  going  from  s 
to  s'  upon  taking  action,  a 

The  status  of  the  system  is  described  by  a  state-vector 
containing  variables  that  adequately  describe  the  evolution 
of  the  system  up  to  a  particular  point  in  time.  If  the 
future  is  known  with  certainty,  i.e.,  it  is  deterministic, 
or  for  that  matter  even  if  the  future  is  uncertain,  an 
optimal  dispatch  strategy  or  policy,  prescribing  what  action 
to  take  from  the  current  state,  can  be  determined. 

In  the  deterministic  case,  any  action  taken  determines 
the  next  state  of  the  system.  In  the  case  where  the  outcome 
of  a  decision  is  uncertain  and  a  probability  distribution 
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for  occupying  a  state  at  the  next  stage  is  known,  the  system 
is  considered  to  be  stochastic  (White  1969,  24).  In  the 
stochastic  case,  probability  transition  matrices  are 
introduced  into  the  dynamic  programming  framework.  They 
describe  the  chances  of  the  system  transitioning  from  s  to 
s'  given  some  action  taken  in  state  s. 

"  With  a  stochastic  dynamic  program,  the  actual  decision 
path  now  depends  on  how  the  random  aspects  of  the  system 
manifest  themselves.  Because  of  this,  "solving"  a 
stochastic  dynamic  program  involves  giving  a  decision  rule 
for  every  possible  state  of  the  HRPS,  not  just  along  a 
single  optimal  path  through  time  which  is  the  result  of 
solving  the  deterministic  case  (Trick  1998). 

For  finite  time  dynamic  programs  (7’<oo)  finding  the 
optimal  solution  is  accomplished  by  evaluating  the  operation 
of  the  system,  one  stage  at  a  time,  starting  with  the  last 
stage  and  working  backwards  in  time.  For  infinite  time 
horizon  dynamic  programs  (T  =  °o  )  other  solution  methods  are 
used  to  find  the  optimal  solution.  Some  of  the  methods 
available  for  solving  infinite  time  MDPs  are  successive 
approximations,  accelerated  successive  approximations  using 
error  bounds,  and  various  forms  of  Policy  Iteration  (Rust 
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1997,  653-658).  Regardless  of  the  time  scale  of  the  dynamic 

program,  the  optimal  policy  (x* )  obtained  is  one  that 
determines  at  each  stage  for  any  state,  a  unique  action  to 
be  taken  from  among  the  allowable  actions  (White  1969,  29) . 
The  policy  is  determined  by  using  Bellman's  principle  of 
optimality  (Bellman  1957).  Succinctly  stated  by  Lapin 
(1985)  as:  "The  optimal  policy  must  be  one  such  that, 
regardless  of  how  a  particular  state  is  reached,  all  later 
choices  proceeding  from  that  state  must  be  optimal." 

3 . 2  Markov  Decision  Processes 

The  problem  considered  here  is  suitably  framed  as  a 
finite,  discrete,  infinite  time  horizon  stochastic  dynamic 
program,  commonly  referred  to  as  a  Markov  Decision  Process. 
Such  a  dynamic  program,  in  addition  to  the  characteristics 
described  in  table  4,  has  a  set  of  transition  probabilities 
that  determine  the  movement  of  the  system  through  its  state- 
space. 

The  MDP  has  a  "finite"  number  of  states,  S=  {1,2,3,..., N} , 
with  transition  probabilities,  P®,  =  p(s'\s,a) ,  from  some 

initial  state,  seS,  to  another  state,  s’  eS ,  being 
dependent  only  on  the  initial  state,  and  an  action,  a, 


31 


taken  from  a  set  of  n  finite  actions,  A  =  {at,a2,a3, , 
available  to  a  decision-maker  in  state  s.  For  this 
framework,  the  number  of  actions  in  any  state  must  be 
finite,  but  the  number  of  actions  available  in  each  state 
may  be  different  from  one  state  to  the  next  (Howard  1960, 

28)  . 

The  state  descriptions  are  an  especially  important 
aspect  of  framing  a  problem  as  a  Markov  Decision  Process. 

The  state  should  inform  us  of  as  much  about  the  environment 
as  possible  in  a  way  that  summarizes  the  relevant  history  of 
the  process  completely  and  compactly.  In  particular,  if 
certain  events  that  have  a  bearing  on  the  decision  to  be 
made  have  happened  prior  to  a  specific  stage  in  the  decision 
process,  then  the  state  description  must  record  these  events 
(White  1969,  32).  A  state  description  that  succeeds  in 
retaining  all  relevant  information  is  said  to  be  Markovian 
(Sutton  1998,  61-62) .  This  means  the  probability  of  moving 
from  one  state  to  another  state  is  independent  of  time. 

This  "memoryless"  property  can  be  assured  with  proper  state 
descriptions  using  a  state-vector  that  captures  the  salient 
features  of  the  system  transitions  up  to  the  point  in  time 
where  an  action  is  taken. 
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If  the  time  between  transitions  is  the  random  variable 
of  interest,  the  system  could  be  considered  a  continuous 
time  process  (Howard  1960,  93) .  Then,  the  significant 
parameters  of  the  HRPS  process  would  be  transition  rates, 
by,  of  going  from  state  i  to  state  j  in  the  time  interval 

dt ,  resulting  in  a  transition  probability  of  bydt ,  (/  *  j)  . 

For  this  model,  however,  the  interest  is  in  the  state 
transitions  due  to  net  load  realizations. 

Since  the  focus  of  this  work  is  on  state  transitions 
based  on  net  load  realizations,  it  is  convenient  to  assume 
that  the  time  between  transitions  is  a  constant  and  we 
therefore  index  state  transitions  in  time  (Howard  1960,  3) . 
Thus,  the  "discrete"  terminology  applies  to  the  stage 
structure;  i.e.,  decisions  are  made  at  discrete  time 
intervals .  ■ 

Analyzing  a  decision  in  terms  of  a  Markov  process  is 
more  valid  in  some  applications  than  others.  In 
applications  where  outside  forces  are  minimal  and  where 
policies  are  expected  to  operate  under  stable  conditions  for 
a  long  time,  an  analysis  based  on  a  Markov  process  is 
appropriate. 
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Stable  conditions  are  assumed  for  this  analysis  of 
hybrid  renewable  power  systems.  By  doing  so,  it  is  assumed 
the  wind  and  sun  resource  profiles  of  a  particular  site 
remain  stable  indefinitely,  barring  any  permanent 
meteorological  changes  at  the  HRPS  site.  In  addition,  given 
the  relatively  simple  lifestyle  of  people  in  remote  villages 
in  developing  countries,  the  electrical  demand  profiles  are 
assumed  to  remain  relatively  stable  over  time.  There  is  a 
caveat  to  this  last  statement,  however.  In  my  opinion,  as 
villagers  become  used  to  availability  of  electrical  power, 
their  demands  for  the  power  may  increase.  Any  significant 
changes  in  electrical  demand  profiles  would  require  re¬ 
running  the  model  to  obtain  a  new  policy. 

I  assume  the  policies  for  energy  dispatch  of  a  HRPS 
under  a  Markov  Decision  Process  framework  face  stationary 
conditions,  i.e.  the  transition  probabilities  are  time 
invariant  and  only  dependant  on  the  state  of  the  system  when 
a  decision  is  made.  Hadley  (1964,  454)  confirms  that  if  the 
distribution  of  the  random  variables  do  not  change  from 
period  to  period,  then  it  is  possible  to  use  an  infinite 
planning  horizon  and  the  influence  of  decisions  at  all 
future  times  are  felt  in  the  current  decision.  More  weight 
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can  be  added  to  this  infinite  planning  horizon  assumption  by 
considering  that  each  of  8,760  hours  in  a  year  over  a 
typical  system  life  of  approximately  20  years  is  evaluated 
in  the  sequential  decision  process.  This  amounts  to  175,200 
time  periods  or  stages,  which  in  my  opinion  is  quite  large 
by  any  planning  horizon  standards. 

If  an  environment  has  the  Markov  property  (Sutton  1998, 
63),  it  follows  equation  3.1, 

P*  =Prto+i  =s’>rM  =r\st,at)  (3.1) 

where  Psss,  is  the  probability  of  going  from  s  to  s’,  under 
action,  a  .  Its  one-step  dynamics  then  enable  us  to  predict 
the  next  state  and  expected  next  cost,  ^P°.C°S. ,  where  C“.  is 

s' 

the  cost  of  going  from  an  initial  state  to  a  successor  state 
under  some  action.  The  Markovian  property  makes  it  is 
possible  to  predict  all  future  states  and  expected  costs 
from  knowledge  of  only  the  current  state,  as  if  we  had  a 
complete  history  of  the  system.  This  implies  the  best 
policy  for  choosing  actions  as  a  function  of  a  Markov  state 
is  just  as  good  as  the  best  policy  for  choosing  actions 
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knowing  the  complete  history  of  the  system  (Sutton  1998, 

63). 

3.3  Model  Specifics 

This  section  describes  the  specific  assumptions  made 
regarding  the  stages,  state-space,  action  set,  cost 
structure,  and  transition  probabilities  in  the  Markov 
Decision  Process  formulation  of  this  model. 

3 .3.1  Stages 

I  assume  an  infinite  time  horizon  with  time  divided 
into  discrete  one-hour  periods  or  stages.  This  is  a  typical 
time  interval  used  for  a  majority  of  modeling  work  with  HRPS 
(Barley  1996;  Lilienthal  1995;  Hancock  1994).  A  one-hour 
time  interval  is  appealing  for  two  reasons.  First,  the 
state-of-charge  is  measured  in  kilowatt-hours  (kWh) . 
Therefore,  in  calculating  the  number  of  kilowatts  removed  or 
added  to  batteries  and  an  appropriate  generator  setting 
measured  in  kW,  a  natural  time  unit  to  employ  is  one  hour. 
The  second  reason  is  dispatch  decisions  for  generators 
should  reflect  some  common  sense  in  terms  of  turning  the 
generator  on  and  off.  For  time  periods  less  than  one  hour, 
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the  constant  starting  and  stopping  of  the  generator,  if  so 
prescribed  by  a  policy,  could  have  an  adverse  affect  on 
engine  lifetime. 

3.3.2  State-Space 

For  the  HRPS,  each  state  in  the  state  space  S  is 
defined  with  the  state-vector  {state-of-charge,  time-of- 
day} .  Figure  3  shows  the  state-space  with  discrete  states- 
of-charge  (y-axis)  ranging  from  0  kWh  to  100  kWh.  The 
state-of-charge  discretization  step,  Asoc ,  is  5  kWh, 
defining  21  possible  states-of-charge :  0,  5,  10,  .  .  .,  100 

kWh.  The  time-of-day  ranges  from  0  (midnight)  to  23  (11:00 
pm) ,  covering  a  24-hour  period.  Consequently,  there  are 
21x24  =  504  states.  Certain  advantages  lie  in  defining  a 
state-space  of  this  size.  The  most  important  of  which  is 
that  for  relatively  small  scale  problems,  where  S' *500,  and 
the  discount  factor,  P  ,  is  sufficiently  close  to  1  (i.e. 

P>. 95)  ,  the  Policy  Iteration  algorithm  is  regarded  as  one  of 
the  fastest  methods  for  computing  the  state-values  and 
finding  an  optimal  policy  (Rust  1996,  654).  The  state-space 
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is  ''fine"  enough  to  give  accurate  and  insightful  results 
without  creating  computational  difficulties.  Even  with  a 
somewhat  coarse  discretization  of  the  state-space,  a  good 
approximation  of  the  underlying  function  can  be  achieved 
(Rust  1996,  659). 

’■  Also,  using  state-of-charge  versus  time-of-day  is 
intuitively  appealing.  When  considering  transitions  from 
one  state  to  the  next,  the  state-space  definition  only 
allows  transitions  from  one  column  to  the  next  in  a  spatial 
sense  (left  to  right  in  figure  3).  With  net  load  as  the 
stochastic  element  in  the  model,  state-of-charge  is  affected 
in  a  vertical  sense  (up  or  down) ,  as  actions  and  net  load 
realizations  charge  or  discharge  the  batteries.  For 
example,  from  state  197  in  figure  3,  the  only  transitions 
available  are  to  states  6,  30,  54,  78,  102,  126,  150, 
174,..., 486.  This  is  true  because  the  time-of-day 
transitions  from  hour  4  to  hour  5  (left  to  right)  with  a 
probability  of  1.  Therefore,  the  set  of  states  to  which  the 
system  can  transition  is  restricted  to  the  column  that 
represents,  time-of-day  =  5.  In  state  197,  the  state-of 
charge  at  the  end  of  hour  4,  soc4 ,  is  60  kWh.  Depending  on 
the  action  taken,  a4 ,  at  the  end  of  hour  4  and  the  net  load 
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realization,  NLS,  over  the  next  hour,  the  state-of-charge 
at  the  end  of  hour  5,  soc5 ,  is  either  higher,  lower,  or  the 
same . 

soc5  =  SOC4  +CI4-  NL5 
if  a4>M,5  then  soc5  >  socA 
else  if  a4  <  NL5  then  soc5<socA 

Another  advantage  of  defining  the  state-space  this  way 
lies  in  the  computational  efficiency  that  can  be  gained  by 
such  a  definition.  The  Policy  Iteration  algorithm  (see 
chapter  3,  section  5)  requires  a  complete  sweep  of  the 
entire  state-space  calculating  expected  values  for  each 
state.  This  equates  to  making  calculations  for  each  of  the 
504  states  for  each  policy  evaluation  and  policy  improvement 
step.  With  the  state-space  defined  here,  calculations  need 
only  be  made  for  21  of  504  or  4%  of  the  states  in  each 
sweep.  This  significantly  reduces  computation  time,  as  96% 
of  the  states  in  the  state-space  do  not  have  to  be  evaluated 
in  each  iteration. 

Furthermore,  defining  the  state-space  with  the 
variables  state-of-charge  and  time-of-day  results  in  a 
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simple  table,  the  interior  of  which  contains  the  optimal 
action  that  an  operator  should  take  given  any  combination  of 
the  two  state  variables.  The  operator  simply  checks  the 
time-of-day  and  a  gauge  measuring  the  state-of-charge  of  the 
batteries,  and  uses  the  table  to  determine  the  appropriate 
generator  setting.  Such  a  process  avoids  even  the  simplest 
mathematical  calculations  that  might  be  involved  with  the 
use  of  a  state-space  that  incorporates  additional  state 
variables. 

This  is  an  important  aspect  of  this  model.  While 
relatively  parsimonious  from  a  modeling  standpoint,  the 
simplicity  cannot  be  overstated.  In  the  event  that  manual 
dispatch  of  a  HRPS  is  needed,  such  a  table  provides  a 
simple,  and  in  my  opinion,  understandable  alternative  to 
automated  dispatch.  With  little  or  no  technical  training, 
even  the  most  inexperienced  person  could  operate  the  system 
manually. 

3.3.3  Action  Set 

There  are  essentially  two  actions  available  to  an 
automatic  controller  or  a  system  operator:  to  leave  the 
generator  off  or  turn  the  generator  on.  If  the  generator  is 
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turned  on,  the  decision  as  to  what  power  level  to  set  it  at 
must  be  considered.  For  this  model  I  assume  discrete 
generator  levels  ranging  from  0  kW  to  100  kW.  The  generator 
discretizaton  step,  Aa ,  is  equal  to  5kW,  matching  the 
discretization  step  for  the  state-of-charge,  Asoc . 
Consequently,  there  are  21  actions  0,  5,  10,  .  .  . ,  100  kW 
available  in  each  state.  By  setting  Aa  =  Asoc  =  ANL  (the  net 
load  discretization  step) ,  this  ensures  that  any  combination 
of  initial  state-of-charge,  action,  and  net  load  realization 
puts  the  system  in  one  of  the  states  defined  previously. 

3.3.4  Cost  Structure 

The  cost  structure  in  this  model  is  composed  of  costs, 
Cass. ,  incurred  as  a  result  of  taking  a  specific  action  and 

transitioning  from  s  to  s' .  There  are  three  types  of  costs 
that  can  be  incurred:  diesel  generation  costs,  battery  wear 
costs,  and  loss  of  load  costs. 

Diesel  generation  costs  are  incurred  any  time  the 
generator  is  turned  on.  They  are  calculated  using  a  linear 
diesel  fuel  consumption  function  (equation  3.2)  defined  by 
Skarstein  and  Uhlen  (1989,  72-87) .  Skarstein  and  Uhlen 
looked  at  several  models  of  diesel  generators  and  developed 
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a  linear  relationship  between  the  rated  power  of  each 
diesel,  pr ,  and  its  fuel  consumption  characteristics. 

F  =.0841 5pr  +.246/?  liters/kWh  (3.2) 

Here  pr  is  the  maximum  generator  power,  fixed  at  100  kW  for 
this  model,  and  p  is  the  power  output  in  any  particular 
hour.  The  fuel  consumption  function  is  multiplied  by  a  fuel 
cost  (cy),  say  $. 264/liter,  to  arrive  at  a  fuel  cost  per 

kilowatt  hour: 

FuelCost  =  cfF  =  .264[.0841 5(100)  +  .24 6p] 

=  222+. 0649 p  (3.3) 

In  addition  to  the  fuel  cost,  an  operation  and  maintenance 
(O&M)  cost  must  be  considered  when  the  generator  is  turned 
on.  I  assume  this  cost  is  $1  per  hour  of  operation 
(Lilienthal,  1998).  For  example,  the  cost  of  running  the 
generator  at  50kW  for  one  hour  would  be 

FuelCost  =  2.22  +  .0649(50)  =  $5,465 


0&M  =  $  1 
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TotalGeneratorCost  =  5.465  + 1  —  $6,465  (3.4) 

Battery  wear  costs  (c^w)  are  incurred  when  the  net  load 
is  positive  (meaning  electrical  demand  is  greater  than  the 
available  renewable  resources)  and  the  generator  is  not 
turned  on.  Battery  degradation  is  actually  a  non-linear 
function  of  the  state-of-charge  at  the  time  the  battery  is 
charged  or  discharged  (Barley  1993) .  For  this  model  I 
assume  that  the  battery  wear  costs  are  a  linear  function  of 
the  change  in  the  state-of-charge  and  are  applied  when  the 
battery  is  discharged.  This  is  a  typical  assumption  used  in 
modeling  HRPS  (Lilienthal  1995;  Barley  1996) . 

The  "loss  of  load"  cost  is  incurred  when  the  action 
taken  in  combination  with  the  state-of-charge  is  not  enough 
to  satisfy  a  positive  net  load.  This  leads  to  unmet 
electrical  demand.  The  loss  of  load  cost  or  penalty  serves 
two  purposes.  Most  importantly,  in  any  optimization  model 
where  minimization  is  the  objective  and  a  lower  bound  is  not 
specified,  the  decision  variables  will  be  set  to  zero.  This 
results  in  a  trivial  solution  to  the  problem. 

The  loss  of  load  cost  or  penalty  can  also  be  thought  of 
as  a  measure  to  ensure  some  level  of  service  to  the 
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consumer.  It  ensures  that  electricity  is  provided  to  the 
greatest  extent  possible,  depending  on  the  magnitude  of  the 
penalty  and  the  stochastic  nature  of  the  net  loads.  For 
developing  countries,  and  specifically  small  villages,  the 
cost  of  unmet  electrical  demand  is  difficult  to  quantify,  so 
sensitivity  analysis  is  performed  to  determine  an  effective 
range  of  penalties.  Preliminary  model  verification  with  a 
deterministic  net  load  profile  indicated  that  a  loss  of  load 
penalty  on  the  order  of  $.30  is  sufficient  to  avoid  loss  of 
load  altogether.  Model  verification  also  indicated  that 
penalties  greater  than  $.75  did  not  change  the  optimal 
strategy,  all  other  parameters  being  equal.  For  the 
stochastic  model,  I  broaden  the  range  and  analyze  the  effect 
of  different  costs  per  kilowatt-hour . ranging  from  $.25  to 
$1. 

3.3*5  Transition  Probabilities 

The  distribution  of  net  loads  ( NL )  that  could  be 
realized  after  an  action  is  taken  determine  the  transition 
probabilities  of  going  from  one  state  to  the  next.  The 
stochastic  nature  of  the  net  loads  is  a  result  of  the 
combination  of  electrical  demand,  available  wind  resource. 
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and  available  sun  resource.  After  an  action,  at  ,  is  taken, 
the  realization  of  the  net  load,  NLt+l,  interacts  with  the 
state-of-charge,  soct ,  to  either  charge  or  discharge  the 
batteries. 

For  this  model,  I  assume  that  the  net  load  profiles  for 
each  hour  of  the  day  are  exogenous  to  the  model.  My 
assumption  is  that  the  profiles  are  provided  by  the  customer 
and  contain  site  specific  data  for  a  potential  HRPS  site. 
Contributions  to  the  net  load  profiles  by  sun  and  wind 
resources  can  be  gathered  directly  or  estimated  using  sun 
and  wind  resource  maps.  The  electrical  demand  contribution 
can  be  estimated  for  potential  sites  using  villages  of 
similar  size,  or  for  sites  where  diesel  generators  already 
exist,  electrical  demand  can  be  measured  directly.  The 
three  components  of  net  load  combine  to  produce  a  net  load 
distribution.  My  framework  accepts  net  load  distributions 
with  any  combination  of  scale  or  location  parameters  as  long 
as  they  can  be  appropriately  discretized.  Here,  I  assume 

each  profile  is  normally  distributed,  NL~N(ju,c r2),  with  a 

prescribed  mean  (  /j  )  and  variance  (  <j2  )  .  I  consider  net 
loads  in  the  range: 


Mnl  6ctnl  ^  NL  <  fj.Nl  +  6<jnl 
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for  each  hour  of  the  day.  Such  an  assumption  captures 
virtually  100%  of  the  possible  realizations  of  the  net  load. 
I  discretize  each  distribution  using  a  method  similar  to 
that  used  by  Rust  (1996,  659) .  I  calculate  probability 
masses  by  taking  the  cumulative  probability  at  a  point  one- 

half  the  net  load  discretization  step,  —ANL,  above  the  net 

load  minus  the  cumulative  probability  at  a  point  one-half 
the  net  load  discretization  step  below  the  net  load.  Two 
exceptions  occur  at  net  loads  closest  to  fiNL  ±  6crNL  .  For 
HNL  -  6<jnl  the  probability  mass  is  one-half  the  net  load 
discretization  step  above  the  net  load  and  includes  the  tail 
to  the  left  of  the  net  load.  For  IUnl  +  6gnl  the  probability 
mass  is  one-half  the  net  load  discretization  step  below  the 
net  load  and  includes  the  tail  to  the  right  of  the  net  load. 
This  approach  ensures  the  realization  of  each  net  load  is 
approximately  centered  on  its  probability  mass  and  the  sum 
of  the  probabilities  over  the  possible  net  loads  sum  to  1. 

To  model  physical  limitations  of  the  batteries,  in 
cases  where  the  state-of-charge,  action,  and  net  load  would 


47 


put  the  system  in  states  that  are  higher  than  battery 
capacity  or  lower  than  zero  charge,  I  accumulate  probability 
at  the  state  closest  to  the  point  where  the  over  or  under 
charge  occurs.  For  example,  if  the  state-of-charge  is  100 
kWh  (maximum  state-of-charge  for  this  model),  the  action 

taken  is  «  =  100  kW,  and  NL  ~  iV(20,42)  with  ANL  =  5  kW,  the 
highest  possible  net  load  under  the  assumptions  I  make  is  40 
kW.  Thus,  the  starting  state-of-charge  plus  the  action 
taken  minus  the  highest  possible  net  load  of  40  kW  over  the 
hour,  results  is  a  theoretical  state-of-charge  of  160  kWh. 

mkWh  +  mkWx\hr-40kWx\hr  =  160kWh 

This  is  clearly  greater  than  the  highest  allowable 
state-of  charge  (100  kWh)  for  this  model,  so  the  probability 
of  being  in  a  state-of-charge  of  100  kWh  is 

40 

'ZPi=Psoc= 100 =1  for  7  =  0, 5,...,  40  (3,5) 

/= 0 

where  is  the  probability  that  the  net  load  is  /.  For 
this  example,  any  realization  of  the  net  load  results  in  a 
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state-of -charge  of  lOOkW.  Similar  calculations  occur  at  the 
lower  boundary  of  the  defined  state-space. 

3 . 4  Solution  of  the  Model  using  Policy  Iteration 

Policy  Iteration  is  an  algorithm  developed  by  Howard 
(1960,  42)  that  converges  on  a  optimal  action  for  each  state 
as  the  number  of  stages  becomes  very  large.  Consequently,  it 
may  be  applied  only  to  continuing  processes  or  to  those 
whose  termination  is  remote.  Policy  Iteration  has 
simplicity  of  form  and  interpretation  that  makes  it  very 
desirable  from  a  computational  point  of  view  and  often 
converges  in  surprisingly  few  iterations  (Sutton  1998,  97). 
Policy  Iteration  has  the  following  properties  (Howard  1960, 
39)  : 

•  The  solution  of  the  sequential  decision  process  is 
reduced  to  solving  sets  of  linear  simultaneous  equations 
for  the  "value"  of  each  state,  where  value  is  a  measure 
of  all  future  costs  to  be  expected. 

•  Each  succeeding  policy  found  in  the  Policy  Iteration 
cycle  finds  a  higher  value  for  a  state  than  the  previous 
policy. 
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•  The  iteration  cycle  will  terminate  on  the  policy  that  has 
the  largest  value  attainable  within  the  scope  of  the 
problem. 

Policy  Iteration  (equation  3.6)  is  composed  of  two 
parts,  a  policy  evaluation  operation  ( — — » )  and  a  policy 
improvement  routine  ( — )  .  The  policy  evaluation  operation 
yields  state-values  ( VK )  as  function  of  policy  ( n )  by 
solving  the  Bellman  Equations,  V’r(s)  = 

s' 

whereas  the  policy  improvement  routine  yields  the  policy  as 
a  function  of  the  state-values.  Policy  Iteration  converges 

to  an  optimal  policy  ( n  )  and  optimal  value  function  ( V* ) 
in  a  finite  number  of  iterations. 

E  I  E  I  E  I  E 

Kq-^V710  it  -> V  (3.6) 

3.4.1  Policy  Evaluation 

Policy  evaluation,  accomplished  by  solving  a  system  of 
S  equations  in  S  unknowns,  computes  the  state-value 
function  V^is)  for  each  state  under  an  arbitrary  policy,  n . 
The  value  functions  derived  from  policy  evaluation  estimate 
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how  good  it  is  to  be  in  a  given  state.  The  notion  of  "how 
good"  is  defined  in  terms  of  future  costs  that  can  be 
expected.  The  future  costs  depend  on  what  actions  are  taken 
so,  the  value  functions  (equation  3.7)  are  defined  with 
respect  to  particular  policies  where  a  policy  represents  the 
mapping,  tt:S->A. 

r'«=£>j|c£+/r'(*'>]  0.7) 

s' 

Equation  3.7,  the  Bellman  equation  for  a  value  function 
V*(s)  ,  expresses  a  relationship  between  the  value  of  a  state 
and  the  values  of  its  successor  states.  There  are  504 
Bellman  equations  for  this  model,  one  for  each  state. 
Equation  3.7  states  that  the  value  of  the  initial  state 

V*(s)  must  equal  the  discounted  value  of  sum  over  the 
expected  next  state-values,  V*(s'),  plus  the  expected  cost 

along  the  way.  In  an  economic  sense,  the  difference 

s' 

in  relative  values  between  two  states  is  equal  to  the  amount 
that  a  rational  person  would  be  willing  to  pay  in  order  to 
start  his  transitions  from  one  state  as  opposed  to  another. 
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These  relative  values  hold  the  key  to  finding  a  better  and 
ultimately  the  best  policy  (Howard  1960,  34-36). 

3.4.2  Policy  Improvement 

For  Markov  Decision  Processes,  we  can  precisely  define 
an  optimal  policy  using  policy  evaluation  under  different 
policies.  Value  functions  define  an  ordering  over  policies, 
consequently,  there  is  always  at  least  one  policy  that  is 
better  than  or  equal  to  all  other  policies.  This  is  the 

optimal  policy,  n  .  It  has  the  state-value  function, 
called  the  optimal  state-value  function,  V*  ,  defined  as 
V*  =ma xV*(s)  for  all  seS.  This  leads  to  the  Bellman 

.  71 

optimality  equation 

K,(5)  =  max^.[C+^(0]  (3.8) 

a  t 

s 

Simply  stated,  any  policy  that  is  "greedy"  with  respect 
to  the  optimal  value  function  is  an  optimal  policy.  The 
term  "greedy"  describes  policies  that  select  actions  based 
only  on  their  short-term  consequences.  The  nice  thing  about 
the  Bellman  Optimality  equations  is  that  if  they  are  used  to 
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evaluate  the  short-term  consequences  of  actions,  then  a 
greedy  policy  is  actually  optimal  in  the  long-term  sense. 

This  is  true  because  V *  takes  into  account  the  cost 
consequences  of  all  possible  future  behavior.  By  means  of 

V* ,  the  optimal  expected  long-term  value  is  turned  into  a 
quantity  that  is  locally  and  immediately  available  for  each 
state.  Hence  the  one-step  ahead  search  yields  the  long-term 
optimal  actions  (Sutton  1998,  75-77),  so  in  theory,  solving 
the  Bellman  optimality  equation  provides  the  route  to 
finding  an  optimal  policy  (Sutton  1998,  79) . 

3 . 5  Policy  Iteration  Algorithm 

Theoretically,  the  simultaneous  solution  of  the  Bellman 
optimality  equations  provides  the  long-term  values  for  each 
state  in  the  state  space.  In  practice  this  requires  solving 
up  to  504  equations  in  504  unknowns  for  this  model.  To 
accomplish  this  task  an  algorithm  is  used  that  allows  for 
the  solution  of  a  system  of  linear  equations  iteratively 
(figure  4).  This  in-place  algorithm  is  a  Gauss-Siedel-style 
algorithm  (Sutton  1998,  110)  for  solving  such  a  system  of 
equations.  Initially,  an  arbitrary  value  and  an  arbitrary 
policy  are  assigned  to  each  state.  Under  the  policy 


1.  Initialization 

F(,s)€9?and  n(s)eA(s)  arbitrarily  for  all  seS 

2.  Policy  Evaluation 

A  <—  0 

For  each  seS: 
v  <-  V(s) 

v(si  <-  + pv’{,)  wj 

s' 

A  <-  max(  A  ,\v-V  (»v)| ) 

until  A<£(a  small  positive  number) 

3.  Policy  Improvement 
"policystable"<-  true 
For  each  s  eS  : 

b  <—  n(s) 

rtfi)*-  argmax£/5lC  +  /ry)J 

a  s' 

If  b*n{s),  then  "policystable"<--/a/.s'e 
If  "policystable",  then  stop;  else  go  to  step  2 

Figure  4.  Policy  Iteration  Algorithm  (using  iterative 

policy  evaluation)  for  V*  .  Source:  Sutton,  page 
98 
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evaluation  operation  the  algorithm  stores  the  initial  values 
of  each  state  and  then  sweeps  through  the  entire  state  set 
calculating  the  long-term  value  of  each  state  under  a 
specific  policy.  It  does  so  until  the  absolute  value  of  the 
maximum  difference  between  the  initial  value  and  the 
calculated  value  of  each  state  is  less  than  e  .  e  is  a  user 
defined  tolerance  that  represents  the  maximum  difference 
that  the  user  is  willing  to  accept  between  the  actual  state- 
value  and  an  approximation  of  the  state-value. 

Under  the  policy  improvement  routine  the  algorithm 
stores  the  current  action  for  each  state  (i.e.,  the  policy, 
n:S->A)  and  then  sweeps  through  the  entire  state  set 
calculating  the  long-term  expected  value  of  each  state  over 
the  entire  action  set.  It  then  picks  the  action  that  gives 
the  highest  value  to  each  state.  This  new  action  is 
compared  to  the  initial  action.  If  there  is  no  difference 
between  the  two  state-values,  then  this  is  the  optimal 
policy  for  that  state. 

I?  is  a  factor  used  to  weight  future  value  relative  to 
the  present.  Its  primary  purpose,  void  of  any  other 
meaning,  is  to  ensure  that  state-values  converge  to  a  finite 
value.  In  fact,  as  long  as  fi<\  the  existence  and 
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uniqueness  of  the  state-value  functions  V *  is  guaranteed 
(Sutton  1998,  90)  .  If  J3  is  greater  than  or  equal  to  1  then 
the  sum  of  present  and  future  costs  would  grow  without 
bound. 

A  common  interpretation  of  J3  is  as  a  discount  factor 
(Winston  1994;  Howard  1960;  Rust  1996).  If  a  person  values 
the  future  a  great  deal  he  or  she  assigns  a  discount  factor 
close  to  1 .  If  a  person  is  only  concerned  with  the  present, 
a  discount  factor  of  0  is  appropriate.  /?  for  a  single 
period  transition  is  calculated  using  the  following  formula 


where  r  is  a  discount  rate.  An  appropriate  discount  rate 
is  extremely  difficult  to  determine  a  priori,  especially  on 
projects  such  as  hybrid  renewable  power  systems. 

While  policy  iteration  can  only  be  applied  to 
continuing  processes  whose  termination  is  remote,  there  is 
still  some  possibility  that  the  process  might  stop.  In 
cases  where  there  is  uncertainty  concerning  the  duration  of 
the  process,  /?  is  considered  to  be  the  probability  that  the 


56 


process  will  continue  to  incur  costs  after  the  next 
transition  (Howard  1960,  77).  This  implies  1-J3  is  the 
probability  that  the  process  will  stop  at  its  present  stage. 
This  is  the  interpretation  used  here. 

For  this  model  I  use  ft  =.999  .  This  is  a  value  of 
that  provides  a  reasonable  measure  in  terms  of  system 
reliability,  while  at  the  same  time  keeping  model  run  times 
to  a  reasonable  length.  With  fi=.999  average  run  times  for 
each  scenario  are  around  4  hrs  on  a  Pentium  166  MHz,  IBM 
compatible  platform  using  Visual  Basic  for  Applications. 

For  /?=.9999,  one  scenario  solved  in  approximately  17  hours 
while  another  scenario  was  hot  done  solving  after  30  hours. 
Another  reason  ft  =.999  is  an  adequate  choice  is  that  as  /? 
approaches  1,  which  Rust  (1996,  631)  defines  as  any  /?>. 95, 
the  state-values  converge  to  those  which  would  be  calculated 
under  the  long-run  average  cost  MDP  formulation: 


.  cos  t  incurred  during  periods!, 2, 3, 
max£[lim( - 

n->co  Yi 


This  implies  /?  =.999  gives  results  that  are,  for  all  intents 
and  purposes,  equal  to  those  that  would  be  found  under  a 
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solution  method  that  does  not  account  for  some  probability 
of  system  failure. 

3.6  Solution  Procedures 

I  use  the  Policy  Iteration  algorithm  to  determine  the 

optimal  policies  (O  for  dispatching  HRPS  for  two 
different  net  load  profiles  under  nine  different  scenarios 
each.  Comparisons  are  made  across  different  fuel  costs, 
battery  wear  costs,  and  loss  of  load  penalties. 

The  first  net  load  profile  (figure  5)  has  positive  net 
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Figure  5.  Net  Load  Profile  1 
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load  values  for  each  of  the  24  hours  of  the  day.  In  hours 
18  and  19,  all  possible  realizations  of  the  net  load  exceed 
the  maximum  generator  rating  (100  kW) .  In  hour  20,  some 
realizations  of  the  net  load  distribution  exceed  the  maximum 
generator  rating.  Both  of  these  situations  may  potentially 
lead  to  a  loss  of  load. 

In  addition,  most  of  the  potential  net  load 
realizations  exceed  the  critical  load,  Lc ,  in  each  of  the  9 
scenarios  (see  chapter  3,  section  7  for  a  complete 
description  of  the  meaning  of  Lc) .  This  means  the  results 
of  the  combined  dispatch  strategy  are  the  same  as  the  load- 
following  strategy.  In  general,  the  individual  net  load 
distributions  do  not  show  wide  variation  around  their  means. 

The  second  net  load  profile  (figure  6)  is  characterized 
by  some  occurrences  of  mean  net  loads  that  are  less  than 
zero,  meaning  the  batteries  in  the  system  should  be  allowed 
to  charge,  free  of  generator  power,  in  those  instances. 

There  are  no  instances  where  any  one  net  load  realization 
exceeds  the  maximum  power  rating  of  the  generator,  so  loss 
of  load  should  be  minimized  all  other  things  being  equal.  In 
addition,  the  variances  around  the  individual  means  are 
quite  large  in  some  instances  and  quite  small  in  others. 
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Figure  6.  Net  Load  Profile  2 

The  potential  net  load  realizations  vary  around  the 
critical  load,  Lc,  in  each  of  the  9  scenarios,  so  the 
combined  dispatch  strategy  may  or  may  not  take  the  form  of 
one  of  the  other  standard  strategies. 

The  various  scenarios  over  which  the  net  load  profiles 
are  examined  and  the  resulting  optimal  policies  developed 
are  shown  in  table  5.  The  base  case  and  ranges  of  parameter 
values  with  the  exception  of  the  loss  of  load  penalty  are 
taken  from  Barley  (1996,  53) .  A  loss  of  load  penalty  of  $1 
is  assigned  as  the  base  case  parameter. 
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Table  5. 

Scenarios 

Scenario 

Number 

Battery  Wear 
Cost  ($/kWh) 

Fuel  Cost 
($/liter) 

Loss  Penalty 
( $ / kWh ) 

Analysis 
of : 

1 

.10 

.264 

1 

Base 

Case 

2 

.10 

.264 

0 

Loss 

3 

.10 

.264 

.25 

Loss 

4 

.10 

.264 

.50 

Loss 

5 

.05 

.264 

1 

Battery 

Wear 

6 

.20 

.264 

1 

Battery 

Wear 

7 

.10 

.10 

1 

Fuel 

Cost 

8 

.10 

.20 

1 

Fuel 
.  Cost 

9 

.10 

.50 

1 

Fuel 

Cost 
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The  different  scenarios  represent  variation  over 
different  parameters.  Runs  1  through  4  vary  the  loss  of 
load  penalty.  Runs  1,  5  and  6  vary  the  battery  wear  cost 
(cbw) ,  and  runs  1  and  7  through  9  vary  the  fuel  cost 
parameter  {cf). 

3 . 7  Evaluation  of  Optimal  Strategies 

After  an  optimal  policy  is  developed  for  a  particular 
scenario,  I  test  the  optimal  policy  versus  the  load¬ 
following,  cycle-charging,  and  combined  strategies  as  well 
as  a  random  strategy.  This  is  done  using  a  simulation  to 
generate  realizations  of  the  net  load  profiles  for  each  hour 
of  the  day.  These  realizations  interact  with  the  state-of- 
charge  of  the  batteries  and  the  action  taken  according  to 
the  specific  dispatch  strategy  under  consideration.  The 
cost  structure  used  in  the  simulation  is  identical  to  the 
one  used  in  development  of  this  framework,  so  a  comparison 
of  the  optimal  dispatch  strategies  to  the  other  dispatch 
strategies  is  consistent  with  the  assumptions  used  in  the 
MDP . 


62 


For  the  optimal  strategy,  the  state-of-charge  and  time- 
of-day  combination  is  referenced  to  the  optimal  dispatch 
table  and  the  prescribed  action  is  taken. 

Under  a  load-following  strategy,  if  the  batteries 
cannot  meet  the  net  load,  diesel  power  is  used  to  just  meet 
the  net  load  and  the  batteries  are  never  charged  by  the 
diesel.  Thus,  the  state-of-charge  does  not  change  in  a 
period  where  the  generator  is  turned  on. 

Under  a  cycle-charging  strategy,  if  the  batteries 
cannot  meet  the  load,  the  diesel  is  run  at  maximum  power, 
where  generators  are  most  efficient,  to  both  meet  the  load 
and  charge  the  batteries.  The  intent  of  this  strategy  is  to 
run  the  generator  at  full  power  and  to  top  off  the  batteries 
with  any  power  in  excess  of  net  load. 

Under  a  combined  strategy,  if  the  batteries  cannot  meet 
the  net  load,  either  a  load-following  or  cycle-charging 
strategy  is  employed.  With  this  strategy  a  critical  load, 

Lc,  is  calculated  for  each  scenario  as  follows: 


8.4 15c, 
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4  is  the  net  load  where  the  cost  of  cycled  diesel  energy 
(denominator),  i.e.  that  which  is  put  into  the  battery  by 
the  generator  and  later  extracted,  is  equal  to  the  cost  of 
direct  diesel  energy  (numerator) .  For  net  loads  less  than 
Lc,  it  is  more  expensive  on  a  $/kWh  basis  to  load-follow  due 
to  generator  inefficiencies,  thus  the  prescription  is  to 
cycle-charge  and  store  any  excess  energy  for  later  use.  For 
net  loads  greater  than  Lc ,  diesel  efficiencies  are  such  that 
it  is  less  expensive  to  load-follow.  In  this  model,  I 
assume  the  round  trip  efficiency,  is  1,  so 

8.415c, 

.  4  =— - -  (3.9) 

cbw 

An  4  is  calculated  for  each  simulated  scenario  according  to 
equation  3.9  and  is  used  in  the  combined  strategy  according 
to  Rules  1  and  2. 

Rule  1:  if  NL>soc  and  NL>LC  then  load-follow 

if  NL>soc  and  NL<LC  then  cycle-charge 


Rule  2 : 
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Rule  1  states  that  if  the  charge  in  the  batteries  is  not 
sufficient  to  meet  the  net  load  and  the  net  load  is  greater 
than  the  critical  load,  then  load-follow.  Rule  2  states 
that  if  the  charge  in  the  batteries  is  not  sufficient  to 
meet  the  net  load  and  the  net  load  is  less  than  the  critical 
load,  then  cycle-charge. 

Under  the  random  strategy,  realizations  are  drawn  on  a 
uniform  distribution,  U~  [0,100].  This  range  represents  all 
of  the  possible  generator  settings  (rounded  to  nearest  the 
action  discretization  step,  A  a)  .  Since  a  random  strategy  is 
the  most  simplistic  strategy  possible,  it  serves  as  a  useful 
baseline  against  which  all  other  strategies  can  be  compared. 

In  addition  to  using  a  simulation  to  test  the  optimal 
strategies,  HOMER  is  used  as  a  test  platform.  It  should  be 
noted  that  the  model  developed  here  is  based  on  first 
principles  of  dispatch  optimization  and  is  a  general 
framework.  Consequently,  its  configuration  is  not  fully 
compatible  with  the  underlying  assumptions  contained  in  a 
specific  model  like  HOMER.  One  such  difference  lies  in  the 
application  of  the  loss  of  load  penalty  measured  in  dollars 
versus  the  "percent  of  unmet  load"  criterion  used  by  HOMER. 
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In  HOMER  the  percent  of  unmet  load  is  an  absolute 
constraint  that  screens  out  any  strategies  that  exceed  it. 
The  percent  unmet  load  is  independent  of  any  dispatch 
decision  and  is  an  indicator  of  what  happened  under  a 
particular  strategy.  This  is  in  contrast  to  the  loss  of 
load  penalty  in  the  general  framework,  which  is  a  tool  for 
making  proactive  dispatch  decisions  to  avoid  loss  of  load. 

Another  difference  between  the  configuration  of  the 
model  developed  here  and  that  of  HOMER  lies  in  the  point  at 
which  a  dispatch  decision  is  made.  The  general  framework 
assumes  that  the  current  state-of-charge  and  time-of-day 
capture  all  relevant  information  about  the  system  up  to  the 
point  in  time  where  a  dispatch  decision  needs  to  be  made 
(see  figure  7).  The  optimal  strategies  are  proactive  in  that 
the  decisions  they  make  anticipate  realizations  of  the  net 
load  Over  not  only  the  next  hour,  but  all  subsequent  hours 
as  well.  HOMER  makes  dispatch  decisions  as  a  reaction  to 
the  current  net  load  only  and  assumes  that  the  net  load  will 
be  the  same  over  the  entire  hour.  Furthermore,  HOMER's 
decisions  are  made  regardless  of  net  load  beyond  the  current 
hour  (see  figure  8) .  Despite  these  configuration 
differences,  the  optimal  dispatch  strategies  still  do  quite 
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Figure  8.  HOMER  Decision  Process 
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well  when  measured  against  standard  dispatch  strategies  on  a 
net  present  cost  basis.  This  indicates  potential  for  gains 
in  cost  reduction  in  a  specific  model  like  HOMER  with 
appropriate  configuration  adjustments  to  the  general 
framework. 
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Chapter  4 

RESULTS  AND  ANALYSIS 


4 . 1  Simulation  Results 

The  solutions  to  the  general  framework  provide  state- 
values  for  each  member  of  the  state-space  and  a  policy  that 
maps  each  state  to  the  action  that  minimizes  the  long-run 
average  value  or  cost  of  each  state.  Examples  of  the  output 
from  the  model  are  shown  in  figures  9  and  10.  Figure  9 
shows  the  discrete  actions  that  should  be  taken  for  each 
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Figure  9.  Optimal  Actions,  Net  Load  Profile  1,  Scenario  1 
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state-vector  describing  a  combination  of  state-of-charge  and 
time-of-day.  Figure  10  is  a  3-D  representation  of  the  same 
policy  showing  interpolated  actions  for  each  state  not  in 
the  defined  state-space.  Interpolated  actions  are  not  used 
in  this  framework,  so  figure  10  is  only  intended  to 
demonstrate  potential  for  extending  the  results  to  a  denser 
state-space.  The  optimal  policies  are  tested  against  each  of 
the  other  dispatch  strategies  described  in  chapter  3, 
section  7 . 


Figure  10.  Optimal  Action  Profile,  Net  Load  profile  1, 
Scenario  1 
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4.1.1  Analysis:  Net  Load  Profile  1 

Figures  11  and  12  show  a  comparison  of  the  long-run 
expected  cost  earned  under  each  optimal  dispatch  strategy 
relative  to  the  other  dispatch  strategies  tested.  It  is 
clear  from  figure  11  that  the  optimal  strategies  under  each 
scenario  provide  the  lowest  long-run  expected  cost  compared 
to  the  other  4  strategies.  Note  Scenario  2  is  not  included 
in  the  comparisons.  Since  a  loss  of  load  penalty  of  $0  was 
used  in  computing  the  optimal  dispatch  strategy,  the  model 
reported  that  the  battery  should  drain  fully  and  the 
generator  should  remain  off.  This  is  an  intuitive  result 
given  the  model  has  no  incentive  to  minimize  loss  of  load. 
Consequently,  a  long-run  average  cost  of  $0  was  reported 
which  confirms  the  validity  of  the  model's  solution.  Figure 
12  is  a  summary  of  the  results  of  the  dispatch  strategy 
comparison.  GREEN  cells  represent  the  LOWEST  VALUES  under 
each  scenario  and  RED  cells  represent  the  HIGHEST  VALUES 
under  each  scenario.  In  all  cases  a  low  value  is  considered 
better  than  a  high  value  since  the  goal  of  the  optimization 
is  cost  minimization  and  minimisation  of  loss  of  load. 


$20.00 


72 


O  O  Q 

O  O  O  , 

* 

\ — | 

iri  o  in  in 

t-  x-  <n 

x — 1 

in  in 

<D 

U 

(jM/$)  isoo 

3 

Gn 

pepadxg  uny-6uo~| 

fa 

74 


Recall  that  the  long-run  expected  cost  and  associated  annual 
costs  include  a  combination  of  generator  cost,  battery  wear 
costs,  and  loss  of  load  penalties. 

It  is  interesting  to  note  that  the  optimal  strategies 
all  have  the  lowest  overall  costs,  but  may  not  have  the 
lowest  cost  in  each  of  the  separate  cost  categories.  The 
annual  generator  and  battery  wear  costs  that  are  incurred 
under  the  different  scenarios  are  not  necessarily  the  lowest 
under  the  optimal  dispatch  strategies. 

For  net  load  profile  1,  the  generator  and  battery  wear 
costs  are  actually  lower  for  the  load-following  and  combined 
strategy  under  most  of  the  scenarios.  The  optimal 
strategies  sacrifice  some  generator  and  battery  wear  cost  to 
keep  loss  of  load  to  a  minimum.  Even  so,  the  generator  and 
battery  wear  costs  for  the  optimal  strategies  are  only 
marginally  worse  than  the  load-following  and  combined 
strategies.  For  optimal  strategies  the  generator  and 
battery  wear  costs  are  about  are  about  8-10%  better  than  the 
worst  case,  cycle-charging  strategy.  Notice  that  for  any 
non-zero  loss  of  load  penalty  (Scenarios  1,  3,  4  with 
penalties  of  $1,  $.25,  $.50,  respectively  and  identical  fuel 
and  battery  wear  costs) ,  the  long-run  average  cost  is 


75 


essentially  the  same  for  each  scenario.  This  implies  that 
any  non-zero  loss  of  load  penalty  will  suffice,  under  this 
net  load  profile,  to  ensure  that  long-run  average  cost  is 
kept  to  a  minimum.  The  size  of  the  loss  of  load  cost  and 
the  associated  quality  of  service  are  a  function  of  the 
magnitude  of  the  loss  of  load  penalty.  For  net  load  profile 
1,  the  optimal  strategies  happen  to  produce  the  lowest  loss 
of  load  costs  under  all  of  the  scenarios  as  well  as  provide 
the  best  level  of  service  to  the  consumer,  when  compared  to 
the  other  strategies. 

In  addition  to  loss  of  load  cost,  quality  of  service  is 
analyzed  using  two  other  metrics.  They  are  the  number  of 
hours  in  a  year  the  system  cannot  meet  demand  and  the 
average  loss  of  load  measured  in  kilowatts.  For  this  net 
load  profile,  the  optimal  strategies  are  convincing  winners 
in  terms  of  these  metrics  (see  figure  13) .  Under  all  of  the 
scenarios,  but  one,  the  optimal  strategies,  have  the  fewest 
number  of  "outages"  in  a  year  with  the  lowest  average  loss 
of  load  measured  in  kW.  For  a  $1  penalty  and  any 
combination  of  fuel  cost  and  battery  wear  cost,  the  optimal 
dispatch  strategies  for  Scenarios  1  and  5  through  9  provide 
the  fewest  number  of  outages  compared  to  all  other 
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13.  Quality  of  Service  Analysis,  Net  Load  Profile 
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strategies.  Under  Scenario  3,  the  loss  of  load  penalty  is 
$.25/kW,  so  the  incentive  for  the  optimal  strategies  to 
minimize  loss  of  load  is  reduced.  Even  so,  the  optimal 
strategy  is  not  the  worst  in  terms  of  quality  of  service 
under  Scenario  3.  Also,  under  Scenario  3,  the  average  unmet 
load  is  far  lower  than  the  other  strategies. 

4.1.2  Analysis:  Net  Load  Profile  2 

Figures  14  and  15  show  a  comparison  of  the  long-run 
expected  costs  for  each  dispatch  strategy  under  net  load 
profile  2.  The  optimal  strategies  under  each  scenario 
provide  the  lowest  long-run  expected  cost,  but  in  some  cases 
are  only  marginally  better  than  standard  dispatch 
strategies . 

In  particular,  under  Scenarios  5,  8  and  9,  the  combined 
strategy  can  provide  essentially  the  same  long-run  expected 
cost  as  the  optimal  strategy.  Note  though,  that  under  net 
load  profile  2,  the  optimal  strategy  places  more  emphasis  on 
reducing  the  generator  and  battery  wear  costs  than  the  other 
strategies.  In  fact,  the  optimal  strategies  produce  the 
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Figure  15.  Summary  of  Dispatch  Strategy  Results,  Net  Load  Profile 
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lowest  generator  and  battery  wear  costs  over  all  of  the 
scenarios  (see  figure  15) . 

Loss  of  load  is  not  a  criterion  by  which  the  standard 
strategies  make  dispatch  decisions.  While  the  optimal 
strategies  are  trying  to  balance  generator  and  battery  wear 
costs  against  loss  of  load  penalties  for  an  extended  period 
of  time;  the  other  strategies  simply  do  what  is  best  to  meet 
current  load,  regardless  of  the  ramifications  in  subsequent 
time  periods.  For  optimal  strategies  under  this  profile, 
the  loss  of  load  cost  is  generally  somewhere  between  the 
best  strategy  for  minimizing  loss  of  load  cost,  cycle¬ 
charging  and  the  worst  strategy,  load-following. 
Consequently,  there  are  mixed  results  in  terms  of  quality  of 
service  for  the  optimal  strategies. 

The  optimal  strategies,  while  less  successful  in 
avoiding  loss  of  load  costs,  are  generally  not  the  worst 
strategies  in  terms  of  service  quality  (see  figure  16). 

They  do  provide  a  mid-range  level  of  service  in  most  cases, 
but  certainly  do  not  provide  as  good  a  quality  as  the  cycle¬ 
charging  strategy.  This  makes  sense  in  that  the  cycle¬ 
charging  strategy  always  "tops  off  the  batteries"  and  thus 
avoids  some  unexpected  loss  of  load.  This  is  an  especially 
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effective  strategy  for  this  net  load  profile  since  all 
realizations  of  net  loads  are  below  the  battery  capacity. 
Cycle-charging  pays  the  price  in  terms  of  generator  and 
battery  wear  costs,  however.  What  the  optimal  strategies 
lack  in  service  quality  under  this  net  load  profile,  they 
make  up  for  in  minimizing  the  generator  and  battery  wear 
costs . 

4 . 2  HOMER  Results 

The  results  of  using  the  optimal  dispatch  strategies  in 
HOMER  under  net  load  profiles  1  and  2  are  shown  in  tables  6 
and  7,  respectively.  Since  the  loss  of  load  penalty  of  $1 
is  most  effective  in  minimizing  the  loss  of  load  in  most 
circumstances,  and  HOMER  is  indifferent  to  a  loss  of  load 
penalty,  I  only  consider  Scenarios  1  and  5  through  9. 

Table  6  shows  that  for  a  net  load  profile  with  all 
positive  net  load  realizations  and  a  higher  likelihood  of 
loss  of  load,  the  optimal  strategies  have  the  lowest  net 
present  cost  (NPC)  and  a  low  percentage  of  unmet  load. 

Where  a  standard  dispatch  strategy  does  not  appear  indicates 
that  its  percent  of  unmet  load  is  too  large.  The  net 
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present  cost  reported  by  HOMER  includes  only  generator  and 
battery  wear  costs.  The  discounted  costs  reported  by  HOMER 
are  calculated  slightly  differently  than  in  the  general 
framework,  so  the  magnitudes  are  not  reported,  only  the 
ranking  of  the  strategies.  The  results  in  table  6  imply 
that  the  optimal  strategies  not  only  reduce  loss  of  load 
over  the  life  of  the  system,  but  also  used  the  least  amount 
of  generator  and  battery  resources.  Notice  that  the 
percentage  of  unmet  load  is  constant  for  all  of  the  standard 
dispatch  strategies  over  all  of  the  scenarios.  This  is  an 
indication  that  HOMER  makes  its  dispatch  decisions 
independently  of  (i.e.  without  anticipating)  net  load 
realizations;  HOMER's  decisions  are  based  only  on  current 
net  load.  The  percentage  unmet  load  figures  are  different 
for  the  optimal  strategies  under  each  scenario.  This  is  due 
to  configuration  differences  between  the  general  framework 
and  HOMER.  In  the  general  framework,  loss  of  load  is 
anticipated  and  in  some  instances  allowed  if  it  leads  to 
lower  overall  cost.  A  trade-off  is  made  between  the  three 
components  of  cost,  loss  of  load  penalty,  generator  cost  and 
battery  wear  cost,  based  on  their  respective  parameters.  For 
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Table  6.  Results  from  HOMER:  Net  Load  Profile  1 


Scenario 

Ranked  by  Lowest 
Net  Present  Cost 

Percent  Unserved 
Load 

1 

OP 

.5% 

CC 

.22% 

CB 

.22% 

5 

OP 

0% 

6 

OP 

.48% 

CC 

.22% 

CM 

.22% 

7 

OP 

.37% 

CC 

.22% 

CM 

.22% 

8 

OP 

.26% 

CC 

.22% 

CM 

,22% 

9  OP  .07% 


OP-Optimal  Strategy  CC-Cycle-Charge  CM-Combined  LF-Load-follow 
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Table  7.  Results  from  HOMER:  Net  Load  Profile  2 


Scenario 

Ranked  by  Lowest 
Net  Present  Cost 

Percent  Unserved 
Load 

1 

LF 

0% 

CC 

0% 

CM 

0% 

OP 

1.16% 

5 

OP 

2.82% 

CC 

0% 

CM 

0% 

LF 

0% 

6 

LF 

0% 

OP 

3.27% 

CC 

0% 

CM 

0% 

7 

LF 

0% 

OP 

.  98% 

CC 

0% 

CM  ' 

0% 

8 

LF 

0% 

OP 

2.5% 

CC 

0% 

CM 

0% 

9 

OP 

2.9% 

CC 

-  0% 

CM 

0% 

LF 

0% 

OP-Optimal  Strategy 

CC-CycIe-Charge  CM-Combined  LF-Load-f ollow 

86 


net  load  profile  2  (table  7),  the  optimal  strategies  place 
either  first  or  second  in  5  of  the  6  scenarios  considered  in 
HOMER.  In  the  Scenario  1,  where  the  optimal  strategy  placed 
last,  there  is  only  a  2%  difference  in  the  net  present  cost 
of  running  the  system  over  a  20-year  life.  Notice  that  the 
percentage  unmet  load  figures  for  the  standard  strategies 
are  0%.  This  is  not  a  surprising  result  given  that  the  net 
load  realizations  for  this  profile  are  all  below  the 
generator  and  battery  capacities.  The  figures  indicate  that 
the  system  can  at  least  meet  load  in  every  time  step  under 
the  standard  strategies.  The  optimal  strategies  show 
differing  percentage  unmet  load  because  they  attempt  to 
balance  the  three  components  of  cost  in  anticipation  of  net 
load  realizations  in  the  future.  HOMER  is  indifferent  to 
trade-offs  between  loss  of  load  penalties  and  other  costs, 
so  it  does  not  "credit"  the  optimal  strategies  with 
minimizing  cost,  only  with  minimizing  net  present  cost. 

4 . 3  Implementing  the  General  Optimization  Framework 

Whether  or  not  one  should  use  the  general  optimization 
framework  presented  here  is  obviously  a  function  of  the  cost 
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of  implementation.  The  following  costs  might  be  incurred  in 
implementing  the  model: 

•  cost  of  gathering  site  data 

•  analysis  of  data  to  determine  net  load  distributions 

•  training  of  analyst  to  familiarize  him/her  with  the  model 

•  computer  run  time 

•  periodic  re-sampling  of  data  to  ensure  the  stationarity 
assumption  still  holds  (see  discussion  in  chapter  3.3.5) 

In  reality,  the  cost  of  implementing  this  framework  should 
be  minimal  for  the  following  reasons: 

If  we  assume  that  a  site  has  been  selected  for 
installation  of  a  HRPS,  it  would  be  necessary  to  collect  net 
load  data  for  determining  the  proper  mix  of  equipment  to 
use.  Therefore,  the  incremental  cost  of  collecting  site 
data  for  use  in  the  general  framework  would  be  $0  as  it  has 
already  been  collected.  Furthermore,  assuming  some 
screening  model,  such  as  HOMER,  is  being  used  to  field  the 
system,  an  analysis  of  the  net  load  distributions  would  also 
have  been  conducted.  So,  using  the  same  argument  as  above, 
the  incremental  cost  of  analyzing  the  net  load  distributions 
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for  use  in  the  general  framework  would,  again,  be  $0. 
Calculation  of  the  transition  probabilities  is  automated  in 
the  model  presented  here.  So,  initially  familiarizing  an 
analyst  with  the  workings  of  the  model  and  specifying 
appropriate  distribution  scale  and  location  figures,  and 
cost  parameters  would  take,  in  my  estimation,  approximately 
one  and  a  one-half  hours.  This  estimate  came  about  because 
I  explained  how  the  model  worked  to  Tom  Lambert  of  NREL  who 
integrated  it  into  a  research  version  of  HOMER.  It  took  him 
less  than  45  minutes  to  understand  the  entire  model.  For 
safety  I  doubled  this  time  to  account  for  persons  not  of  his 
level  of  understanding  or  education.  Assuming  an  analyst 
makes  $20  per  hour  (Lilienthal  1999),  this  would  amount  to  a 
cost  estimate  of  $30. 

Computer  run  time  for  determining  an  optimal  strategy, 
based  on  actual  run  times  for  18  cases  used  in  this  work,  is 
approximately  4  hours.  If  we  assume  the  analyst  already 
possesses  a  personal  computer  for  doing  site  analysis,  I 
would  estimate  that  the  incremental  cost  of  calculating  the 
optimal  dispatch  strategy  might  be  as  high  as  $1.  In  total, 
the  cost  of  initially  setting  up  the  system  with  an  optimal 
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dispatch  strategy,  as  assumed  above,  would  be  approximately 
$31. 

Periodic  re-sampling  of  the  site  data  would  have  to  be 
conducted  to  ensure  that  the  stationarity  assumption  still 
holds.  Assuming  that  the  system  is  being  dispatched  with 
heuristic  methods,  sensors  would  have  to  be  available  to 
tell  the  controller  the  current  net  load  on  the  system.  It 
is  reasonable  to  assume  that  this  sensor  data  is  already 
being  recorded  since  having  such  data  would  be  useful  in  the 
future  for  fielding  systems  in  villages  of  similar  size  or 
with  similar  weather  conditions.  Periodic  re-sampling 
consists  of:  (1)  determining  the  change  in  net  load 
distributions,  if  any,  (2)  deciding  if  it  is  necessary  to 
re-run  the  model,  and  (3)  supplying  the  new  data  and 
parameters  for  the  model.  For  re-sampling,  I  assume  the 
analyst  is  already  familiar  with  the  model,  so 
familiarization  time  is  replaced  by  the  time  required  to 
determine  any  net  load  distribution  changes.  In  total, 
assume  the  incremental  cost  of  re-sampling  would  be  close  to 
the  $31  estimated  earlier.  Clearly,  the  total  annual  cost 
of  implementing  optimal  dispatch  would  be  on  the  order  of 
$62,  if  we  assume  after  an  initial  run  that  we  re-sample 
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twice  a  year.  It  must  be  noted  that  re-sampling  is  required 
at  some  interval  because  of  the  possibility  that  usage  of 
electricity  would  rise  as  people  become  accustomed  to  its 
availability. 

Any  conclusions  regarding  benefit-cost  analyses  based 
on  the  results  presented  here  are  limited  to  the  net  load 
profiles  examined  here  and  parameter  specifications  made  in 
Scenarios  1  through  9.  The  benefit  of  using  the  optimal 
dispatch  strategies  is  measured  by  the  extent  to  which  the 
optimal  strategies  reduce  the  long-run  average  cost  per  hour 
of  running  the  system  compared  to  the  next  best  heuristic 
method  (see  table  8).  Multiplying  the  hourly  savings  by 


Table  8.  Optimal  Strategies  vs.  Next  Best  Heuristic 


1 

3 

4 

Scenario 

5 

6 

7 

8 

9 

Net  Load  Profile  1 

Next  Best  Heuristic 

$8.06 

(LF/CC/CM) 

$7.27 

(LF/CM) 

$7.54 

(LF/CM) 

$6.97 

(LF/CM) 

$8.20 

(LF/CM) 

$4.42 

(LF/CM) 

$6.64 

(LF/CM) 

$12.51 

(LF/CM) 

Optimal 

$7.14 

$7.77 

$7.16 

$6.64 

$7.38 

$3.45 

$5.71 

$12.22 

Net  Load  Profile  2 

Next  Best  Heuristic 

$4.89 

(CC) 

$4.70 

(CC) 

$4.79 

(CM) 

$4.04 

(CC) 

$5.33 

(LF) 

$2.79 

(CM) 

$4.11 

(CM) 

$7.34 

(CM/CC) 

Optimal 

$4.66 

$4.61 

$4.62 

$4.00 

$5.28 

$2.60 

$4.11 

$7.31 
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8,760  hours  per  year  annualizes  this  benefit.  The  annual 
cost  of  implementing  the  optimal  strategies  is  then  netted 
against  the  annual  cost  savings  to  arrive  at  the  annual  net 
benefit  of  implementing  the  optimal  strategies.  These 
results  are  shown  in  tables  9  and  10.  An  explanation  of 
tables  9  and  10  follows. 

In  table  9,  the  number  in  row  1,  column  1  ($.92)  is  the 
difference  in  long-run  average  cost  between  the  next  best 
heuristic  method  (LF/CC/CM — $8.06)  and  the  optimal  dispatch 
strategy  ($7.14).  Continuing  down  the  first  column  in  table 
9,  this  hourly  cost  savings  is  annualized  by  multiplying 
$.92  by  8,760  hours  per  year  to  arrive  at  an  annual  cost 
savings  of  $8,059  under  Scenario  1.  The  cost  of 
implementation  is  then  subtracted  from  $8,059  to  arrive  at 
an  annual  net  benefit  of  $7,997.  The  annual  net  benefit  is 
inflated  at  a  rate  of  3%  per  year  (Survey  of  Professional 
Forecasters  1999)  over  a  typical  20-year  system  life.  The 
inflated  annual  net  benefits  are  then  discounted  using 
discount  rates  ranging  from  5%  to  20%. 

Under  net  load  profile  1  (table  9),  the  annual  net 
benefit  of  using  the  optimal  dispatch  strategies  versus 
using  the  next  best  heuristic  strategy  in  a  system  ranges 
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from  $814  to  $8,435.  Under  all  discount  rates  ranging  from 
5%  to  20%,  the  net  present  benefit  over  the  20-year  life  of 
the  system  is  positive  under  all  scenarios.  This  implies 
that  it  makes  sense  to  use  the  optimal  strategies  for 


dispatching  systems  facing  a  net  load  profile  similar  to 


Table  9. 

Benefit 

-Cost 

Analysis,  Net 

Load 

Profile 

1 

Net  Load  Profile  1 

1 

3 

4 

Scenario 

5 

6 

7 

8 

9 

Cost 

Savings  ($/hr) 

$0.92 

$0.10 

$0.38 

$0.33 

$0.82 

$0.97 

$0.93 

$0.29 

Annual  Cost  Savings 

$8,059 

$876 

$3,329 

$2,891 

$7,183 

$8,497 

$8,147 

$2,540 

Cost  of 

Implementation/ 

Maintnenace 

$62 

$62 

$62 

$62 

$62 

$62 

$62 

$62 

Annua!  Net  Benefit 

$7,997 

$814 

$3,267 

$2,829 

$7,121 

$8,435 

$8,085 

$2,478 

Discount  Rate 

5% 

$134,057 

$13,645 

$54,761 

$47,419 

$119,373 

$141,399 

$135,526 

$41,545 

10% 

$83,935 

$8,543 

$34,287 

$29,690 

$74,741 

$88,532 

$84,854 

$26,012 

15% 

$60,185 

$6,126 

$24,585 

$21,289 

$53,593 

$63,481 

$60,844  ■ 

$18,652 

20% 

$45,794 

$4,661 

$18,707 

$16,198 

$40,778 

$48,302 

$46,296 

$14,192 

that  of  net  load  profile  1.  Under  net  load  profile  2  (table 
10),  the  annual  net  benefit  of  using  the  optimal  dispatch 
strategies  in  a  system  ranges  from  -$62  to  $1,953.  This 
implies  that  there  may  be  circumstances.  Scenario  8  for 
example,  where  it  is  more  advantageous  to  use  an  existing 
heuristic  dispatch  method.  Under  this  scenario  the  cost  of 
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implementing  the  optimal  strategy  exceeds  any  cost  savings 


that  might  be  realized.  Nevertheless,  under  the  other  7 
scenarios,  a  net  annual  benefit  produces  a  positive  net 
present  benefit  of  using  the  optimal  strategies  over  the 
life  of  the  system.  So,  again,  it  makes  sense  to  use  the 
optimal  dispatch  strategies  in  those  seven  scenarios  for 
systems  facing  net  load  profiles  comparable  to  net  load 
profile  2 . 


Table  10. 

Benefit 

-Cost 

Analysis,  Net 

Load 

Profile 

2 

Net  Load  Profile  2 

1 

3 

4 

Scenario 

5 

6 

7 

8 

9 

Cost 

Savings  ($/hr) 

$0.23 

$0.14 

$0.17 

$0.04 

$0.05 

$0.19 

$0.00 

$0.03 

Annual  Savings 

$2,015 

$1,226 

$1,489 

$350 

$438 

$1,664 

$0 

$263 

Cost  of 

Implementation/ 

Maintnenace 

$62 

$62 

$62 

$62 

$62 

$62 

$62 

$62 

Annual  Net  Benefit 

$1,953 

$1,164 

$1,427 

$288 

$376 

$1,602 

-$62 

$201 

Discount  Rate 

5% 

$32,797 

$19,581 

$23,986 

$4,896 

$6,365 

$26,923 

-$977 

$3,428 

10% 

$20,496 

$12,221 

$14,979 

$3,027 

$3,946 

$16,818 

-$651 

$2,108 

15% 

$14,696 

$8,763 

$10,741 

$2,170 

$2,830 

$12,059 

-$467 

$1,511 

20% 

$11,182 

$6,668 

$8,173 

$1,651 

$2,153 

$9,176 

-$355 

$1,150 

4.4  Conclusions 

Using  a  relatively  parsimonious  model,  the  results  of 
the  simulations  show  that  under  the  net  load  profiles 
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considered  here  the  optimal  dispatch  strategies  minimize  the 
long-run  average  and  total  costs  under  all  scenarios.  It 
has  been  shown  that  optimal  dispatch  strategies  can  reduce 
total  system  life-cycle  cost  where  generator  cost,  battery 
wear  and  loss  of  load  penalties  are  considered. 

For  net  load  profiles,  such  as  net  load  profile  1, 
where  loss  of  load  is  almost  assured,  the  optimal  strategies 
shine.  They  minimize  loss  of  load  while  incurring  only 
marginally  more  generator  and  battery  wear  cost  than  the 
other  strategies.  This -has  a  further  advantage  in  that  it 
implies  it  might  be  possible  to  use  an  undersized  generator 
(one  that  is  not  sized  to  meet  expected  peak  demand)  in  a 
HRPS .  This  would  reduce  initial  capital  costs  as  larger 
generators  are  more  expensive.  For  net  load  profiles  where 
a  wide  range  of  net  load  realizations  can  be  expected  from 
one  hour  to  the  next,  the  optimal  strategies  incur  more  loss 
of  load  in  exchange  for  reducing  generator  and  battery  wear 
costs. 

The  results  of  running  the  optimal  strategies  in  HOMER 
differ  from  those  of  the  simulation.  This  is  to  be  expected 
given  that  the  simulation  is  an  exact  depiction  of  the 
conditions  under  which  the  general  framework  was  developed. 
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Even  though  the  configuration  of  HOMER  is  not  true  to  the 
general  framework,  there  is  enough  evidence  to  suggest  that 
modifications  to  the  framework  can  achieve  improvements  in 
the  way  HOMER  or  any  other  specific  HRPS  model  makes 
dispatch  decisions. 

By  extending  the  state-vector  in  the  model  to  include 
current  net  load  as  an  element,  any  advantages  of  the 
optimal  strategies  have  over  the  standard  strategies  will 
only  be  magnified. 

This  model  simultaneously  considers  generator  costs, 
battery  wear  costs  and  loss  of  load  penalties  in  dispatch 
decisions  where  other  models  do  not.  It  also  gives  the  user 
the  flexibility  to  balance  service  availability  against 
lower  operating  costs. 

Dispatch  decisions  can  be  made  independently  of  initial 
and  final  conditions  of  the  system,  avoiding  potentially 
sub-optimal  decisions  about  battery  charge  levels  or 
generator  settings  at  the  beginning  or  end  of  a  planning 
period.  Also,  in  the  event  of  a  system  stoppage  and 
regardless  of  the  experience  of  the  operator,  optimal 
dispatch  decisions  can  be  made  manually. 
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The  framework  is  predictive  in  nature,  anticipating 
potential  net  load  realizations  in  future  periods  and  the 
ramifications  of  any  immediate  dispatch  decisions.  It  is 
also  flexible  enough  to  consider  any  non-linearities  in  cost 
or  constraint  functions. 
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Chapter  5 

ISSUES  FOR  FURTHER  ANALYSIS 


Now  that  it  has  been  established  that  optimal 
strategies  can  reduce  long-run  average  cost,  an  interesting 
extension  to  this  work  would  be  to  determine  the  types  of 
net  load  profiles  over  which  the  optimal  dispatch  strategies 
are  most  effective.  Here,  one  fairly  standard  net  load 
profile  (Net  Load  Profile  1)  and  another  with  fairly  unusual 
characteristics  (Net  Load  Profile  2)  were  used.  Clearly, 
the  optimal  strategies  are  more  effective  with  Net  Load 
Profile  1  in  reducing  long-run  average  cost  relative  to  the 
standard  strategies  than  they  are  with  Net  Load  Profile  2. 

It  would  be  interesting  to  record  the  variance  of  the  net 
load  profiles  around  the  profile  mean  and  the  variance  of 
the  net  loads  around  their  respective  hourly  means,  then  map 
these  characteristics  against  the  optimal  strategies  and  the 
standard  strategies.  This  would  require  the  analysis  of 
several  more  net  load  profiles  and  require  perhaps  hundreds 
of  hours  more  of  simulation  time. 
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Under  the  assumptions  used  here,  the  state-of-charge 
and  time-of-day  capture  all  of  the  relevant  information  up 
to  the  point  in  time  where  an  action  must  be  taken.  A 
further  extension  of  the  model  would  expand  the  state-vector 
to  include  current  net  load.  Thus,  the  model  would  not  only 
anticipate  future  net  load  realizations,  but  it  would  react 
to  current  system  requirements  as  a  separate  entity  rather 
than  include  it  as  one  realization  out  of  a  set  of  net  loads 
expected  over  the  next  time  period. 

The  disadvantage  of  expanding  the  state-vector  and 
consequently  the  state-space  is  that  the  number  of  states 
grows  as  the  product  of  the  state-of-charge,  time-of-day  and 
the  number  of  new  elements  in  the  current  net  load 
descriptor.  So,  if  the  current  net  load  descriptor  contains 
only  4  elements  for  example,  which  is  an  optimistically  low 
number,  the  size  of  the  state-space  would  grow  to  504  x  4  = 
2,016  states.  Then  solution  algorithms  such  as  Policy 
Iteration  would  take  considerably  longer  on  the  desktop 
computer  platform  used  here.  Other  solution  algorithms 
would  have  to  be  considered,  such  as  modified  policy 
iteration,  reinforcement  learning  or  other  mechanisms 
described  by  Rust  (1996) . 
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Another  extension  would  include  an  appropriate 
allowance  for  non-linearities  in  battery  wear.  When 
batteries  are  charged  and  discharged  from  a  high  state  of 
charge,  they  experience  less  "wear."  However,  it  is 
advantageous  to  periodically  use  most  of  the  charge  in  the 
battery  and  then  fully  charge  it  again.  Conversely,  as 
batteries  are  charged  and  discharged  from  low  state  of 
charge,  they  experience  a  great  deal  of  wear.  Also,  they 
should  not  be  left  in  low  states  of  charge  for  extended 
periods  of  time.  All  of  these  issues  could  be  incorporated 
into  the  cost  structure  of  this  model. 

The  HRPS  considered  here  are  intended  to  provide  power 
for  village  size  populations.  The  concept  could  be  extended 
to  providing  power  to  small,  remote  military  bases.  Most 
remote  military  bases  use  generator  power  to  meet  essential 
and  non-essential  power  requirements.  In  the  event  that 
fuel  availability  is  strictly  curtailed  or  eliminated 
altogether,  a  HRPS  could  be  used  to  meet  essential  power 
requirements.  ■ 
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Appendix  A 

DESCRIPTION  OF  THE  HYBRID  OPTIMIZATION 
MODEL  FOR  ELECTRIC  RENEWABLES  (HOMER) 


Until  the  advent  of  HOMER  either  relatively  simple 
spreadsheet  models  or  very  complex  simulation  models  were 
used  for  the  preliminary  design  of  hybrid  renewable  power 
systems  (Lilienthal  1995,  1) .  This  hindered  the  development 
of  HRPS  because  no  simple  methods  for  determining  the  sizing 
of  the  components  existed.  Spreadsheet  models,  useful  for 
financial  analysis,  cannot  reliably  predict  the  performance 
of  hybrid  systems  with  intermittent  resources  and  storage 
because  they  do  not  consider  the  temporal  patterns  of  loads 
and  resources.  Dynamic  simulation  models  are  useful  for 
evaluating  the  performance  of  specific,  well-defined 
systems,  but  are  very  detailed  and  time  consuming  to  set  up 
and  run. 

HOMER  fills  the  gap  between  these  two  types  of  models 
by  giving  the  user  insight  into  how  the  system  design  is 
affected  by  changes  in  parameters,  such  as  component  costs, 
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technology  performance,  load  management  strategy,  dispatch 
strategies,  and  load  and  resource  profiles.  Since  the 
optimal  size  of  each  component  is  an  output  of  HOMER  and  the 
detailed  simulation  models  require  the  user  specify  the  size 
of  each  component,  there  is  a  natural  flow  of  information 
between  the  models.  When  considering  a  new  site,  HOMER  can 
quickly  determine  a  small  set  of  optimal  or  near-optimal 
systems  for  more  detailed  analysis  with  simulation  models. 

HOMER  requires  two  basic  types  of  input.  The  first 
input  describes  hourly  profiles  of  the  available  renewable 
resources  and  loads  and  the  second  describes  the  components 
of  the  system  (Lilienthal  1995). 

Resources  and  loads  must  be  specified  hourly.  Although 
actual  hourly  data  can  be  used,  a  more  compact 
representation  uses  hourly  profiles  for  typical  days  for 
each  season.  A  season  can  be  any  number  of  consecutive 
weeks.  In  order  to  model  the  stochastic  nature  of  both 
renewable  resources  and  loads,  these  typical  days  are  then 
expanded  into  a  user-specified  number  of  modeled  days  by 
adding  two  types  of  noise  to  the  profiles.  An  hourly  noise 
parameter  independently  perturbs  each  of  the  hourly  values, 
while  a  daily  noise  parameter  perturbs  an  entire  day.  The 
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daily  noise  is  important  because  persistent  weather  patterns 
have  a  major  impact  on  storage  requirements.  For  resource 
profiles,  HOMER  uses  Normal  distributions  that  incorporate 
both  the  variation  inherent  in  the  typical  profile  and  the 
two  noise  parameters.  In  addition  to  the  typical  profiles 
and  noise  parameters,  average  wind  speed  and  average  full- 
sun  hours  are  specified  for  each  season.  Separate 
residential  and  commercial  loads  (electrical  demands)  can  be 
specified  for  weekdays  and  weekends. 

Five  types  of  components  must  be  described  for  HOMER: 
wind  turbines,  photovoltaic  arrays,  batteries,  inverters, 
and  diesel  generators.  Capital  cost  and  lifetime  are 
specified  for  each  component.  Normally,  HOMER  chooses  the 
optimal  size  of  each  component,  but  each  of  these  sizes  can 
be  fixed  if  a  particular  system  is  being  modeled.  All 
components,  except  the  wind  turbines,  are  continuous 
variables.  Wind  turbines  come  in  discrete  units,  so  HOMER 
chooses  the  number  of  each  type  of  turbine.  PV  has  a  fixed 
cost  as  well  as  a  cost  per  kW.  Efficiencies  are  specified 
for  the  inverter  and  diesel  generators.  The  diesel 
parameters  can  be  specified  separately  for  small  and  large 
diesels  to  capture  the  economies  of  scale  in  that 
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technology.  Diesel  operation  and  maintenance  is  specified 
as  a  cost  per  hour  of  operation. 

Batteries  require  special  consideration  when  modeling 
isolated  systems.  HOMER  keeps  track  of  their  state-of- 
charge  and  energy  flow  in  each  hour,  subject  to  constraints 
on  the  rate  at  which  they  can  be  charged  and  discharged  as 
well  as  their  minimum  acceptable  state-of-charge . 
Furthermore,  the  lifetime  of  a  battery  is  a  function  of 
usage,  so  HOMER  charges  depreciation  for  each  kWh  of  battery 
throughput . 

HOMER  outputs  include  hourly  energy  flows  through  each 
component,  the  impact  of  several  dispatch  strategies,  and 
economic  information  such  as  the  cost  of  energy  and  net 
present  cost  of  the  system. 
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Appendix  B 

CODE  FOR  POLICY  ITERATION  ALGORITHM 


Bolded  Lines  are  Comments 

Public  Const  numhours  =  24,  numcharge  =  21,  numactions  =  21, 
epsilon  =  0.1,  beta  =  0.9999,  pr  =  100,  cf  =  0.264,  res  =  5, 
losspen  =  5,  battcost  =0.1 

numhours=number  of  hours ; 
numactions=number  of  actions  available; 
epsilon=accep table  tolerance  for  state  value 
beta=  discount  factor; 
pr=max  power  rating  of  generator; 
cf=cost  of  fuel  per  liter; 

res=resolution  of  state  of  charge  and  actions 
losspen=  loss  of  load  penalty; 
battcost=battery  wear  cost 

Public  v(l  To  numcharge,  1  To  numhours),  pol(l  To  numcharge, 
1  To  numhours),  numStates,  a (numactions) ,  cost,  valuestate(l 
To  (numhours  *  numcharge)),  policystate ( 1  To  (numhours  * 
numcharge)),  timestate(l  To  (numhours  *  numcharge)),  p,  k, 
socstate(l  To  (numhours  *  numcharge)),  penalty,  nl,  wear, 
myprob,  loss,  dump,  OM,  diesel,  iter,  policystable, 
countload(0  To  numhours  -  1),  netload(0  To  numhours  -1,1 
To  100),  probnetload (0  To  numhours  -  1,  1  To  100) ,  ps,  first 

1 *********************************************************** 

Sub  Stochastic () 

first  =  False  Is  this  the  first 

run  with  these 
parameters  ? 


108 


False  means  use 
prior  runs  results 
as  seeds  for  the 
next  run 

numstates  =  numhours  *  numcharge  Total  number  of 

states  in  state 
space 

starttime  =  Time  Marks  starting  time 

iter  =0  Initialize  policy 

eval  and  improvement 
counter  to  zero 

1 *********************************************************** 

If  res  =  10  Then 

Call  loadlO 

Elself  res  =5  Then 
Call  load5 
Else: 

Call  load2 

End  If 

I***************************************  ******************** 

Main  Subroutine 

Call  initialactions 

If  first  =  True  Then 

Call  initialstatepolicy 


Load  netloads  and 
probabilities  based  on 
resolution  being  used 
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Call  mreload  Reload  old 

values  if  this 
is  NOT  first 
run 

linelO:  Call  policyeval 
iter  =  iter  +  1 
Call  policyimprove 

If  policystable  =  False  Then 
GoTo  linelO 
End  If 

★★★★a***************************************************** 

For  i  =  1  To  numcharge 

For  j  =  1  To  numhours 

Worksheets ( "Policy" ). Cells (i,  j  +  2)  = 
policystate ( (i  -  1)  *  numhours  +  j) 

Next  j 

Next  i 

For  i  =  1  To  numcharge 

For  j  =  1  To  numhours 

Worksheets ("Values") .Cells (i,  j  +2)  = 
values t ate ( (i  -  1)  *  numhours  +  j) 

Next  j 

Next  i 


Print  out  policy 
and  values 


Checks  for 
better  policies 
for  each  state 


Solves  Bellman 
Equations 


Else : 


End  If 
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endtime  =  Time  Marks  ending 

time 

Worksheets ("Values") .Cells (1,  1)  =  starttime 
Worksheets ("Values") .Cells (2,  1)  =  endtime 

End  Sub 

i *********************************************************** 


Sub  initialactions ( ) 

For  m  =1  To  numactions 

a (m)  =  (m  -  1)  *  res  Assigns  action 

set  to  a (m) 

Next  m 
End  Sub 


Sub  initialstatepolicy ( ) 


For  i  =  1  To  numcharge 

soc  =  ((100  /  res)  +  1  -  i)  *  res 


For  j  =  1  To  numhours 


t  =  (j  -  1) 

v  ( i ,  j  )  =  0 

pol  (1,  j )  =  a (1) 

k  =  ( (i  -  1)  *  numhours)  +  j 


initializes 
policy  and 
values  if  first 
run 


Calculates  SOC 
for  this  state 

Assigns  initial 
values  and 
policies 

Calculates  time 
for  this  state 

Creates  scalar 
equivalents 


Ill 


valuestate (k)  =  v(i,  j) 


Assigns  values 
and  policies  to 
scalars 


policystate (k)  =  pol(i,  j) 

timestate(k)  =  t  Assigns  soc  and 

time  to  scalar 
states 

socstate (k)  =  soc 
Next  j 
Next  i 
End  Sub 

i *********************************************************** 


Sub  mreload()  Uses  values  and 

policies  from 
previous  run 
as  seeds 

For  i  =  1  To  numcharge  Load  old  policy  and 

values  into  current 
state  space 

For  j  =  1  To  numhours 


policystate ( (i  -  1)  *  numhours  +  j)  = 
Worksheets ("Policy") .Cells  (i,  j  +  2) 

Next  j 

Next  i 

For  i  =  1  To  numcharge 


For  j  =  1  To  numhours 

valuestate ( (i  -  1)  *  numhours  +  j)  = 
Worksheets ("Values") .Cells (i,  j  +  2) 
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Next  j 
Next  i 

For  i  =  1  To  numcharge 

soc  =  ((100  /  res)  +  1  -  i)  *  res  Calculates  soe 

for  this  state 

For  j  =  1  To  nuinhours 


Calculates  time 
for  this  state 


k  =  ( (i  -  1)  *  numhours)  +  j  Creates  scalar 

equivalents 

timestate (k)  =  t  Assigns  SOC  and 

time  to  scalar 
states 

socstate (k)  =  soc 
Next  j 
Next  i 
End  Sub 

»  *********************************************************** 


Sub  policyeval() 
Dim  delta 


Solves  Bellman 
equations 


Open  "c:\p_eval.txt"  For  Output  As  #2  (opens  file  for  data 
storage) 


delta  =0 

For  k  =  1  To  numstates 
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For  each  state 

tempvalue  =  valuestate (k)  save  current  value 

of  state  k 

storevalue  =0 

storeprob  =0 

checkmod  =  k  Mod  numhours 

If  k  <  numhours  And  checkmod  <>  0  Then  Top  row 

For  p  =  (k+1)  To  ( (k  +  1)  +  numhours 

*  (numcharge  -  1) )  Step  numhours 

Call  calcprob  Calculates 

transition 

probabilities 

Call  calccost 

storeprob  =  storeprob  +  myprob 

storevalue  =  storevalue  +  myprob  *  (cost  +  beta  * 
valuestate (p) ) 

If  (k  =  99)  Then  Print  #2,  "s  "  &  k  &  "  s'  "  &  p  &  "  prob  " 
&  myprob  &  "  cost  "  &  cost  &  "  loss  "  &  loss  &  "  expnl  "  & 
nl  &  "  wear  "  &  wear  &  "  value  "  &  storevalue 

Writes  data  to  file  for  analysis 

Application . StatusBar  =  "eval  iter=  "  &  iter  +1  &  "  k=  "  & 
k  &  "  p=  "  &  p  &  "a=  "  &  a  (m) 

Shows  status  of  algorithm 

Next  p  Next  possible 

successor  state 

Elself  checkmod  =  0  Then  Last  column 

For  p  =  1  To  (numhours  *  (numcharge  -  1)  +  1) 
Step  numhours 
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Call  calcprob 
Call  calccost 

storeprob  =  storeprob  +  myprob 

storevalue  =  storevalue  +  myprob  *  (cost  +  beta 
*  valuestate (p) ) 

If  (k  =  99)  Then  Print  #2,  "s  "  &  k  &  "  s '  "  &  p  &  "  prob  " 
&  myprob  &  "  cost  "  &  cost  &  "  loss  "  &  loss  &  "  expnl  "  & 
nl  &  "  wear  "  &  wear  &  "  value  "  &  storevalue 

Application. StatusBar  =  "eval  iter=  "  &  iter  +  1  &  "  k=  "  & 
k  &  "  p=  "  &  p  &  "a=  "  &  a(m) 

Next  p 

Else 

Not  top  row 
or  last 
column 

For  p  =  (timestate (k)  +  2)  To  ( (numcharge  -  1)  * 

numhours  +  (timestate (k)  +  2))  Step  numhours 

Call  calcprob 

Call  calccost 

storeprob  =  storeprob  +  myprob 

storevalue  =  storevalue  +  myprob  *  (cost  +  beta 
*  valuestate  (p) ) 

If  (k  =  99)  Then  Print  #2,  ”s  "  &  k  &  "  s'  "  &  p  &  "  prob  " 
&  myprob  &  "  cost  "  &  cost  &  "  loss  H  &  loss  &  "  expnl  "  & 
nl  &  "  wear  "  &  wear  &  "  value  "  &  storevalue 

Application. StatusBar  =  "eval  iter=  "  &  iter  +  1  &  "  k=  "  & 
k  &  "  p=  "  &  p  &  "a=  "  &  a(m) 

Next  p 

End  If 
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If  Abs (storeprob  -  1)  >0.000001  And  Abs (storeprob  -  1)  <>  1 

Then  Stop  Checks 

probabilities 
add  to  1 

End  If 

valuestate ( k)  =  storevalue 

If  Abs (tempvalue  -  valuestate (k) )  >  delta  Then 

delta  =  Abs (tempvalue  -  valuestate (k)) 

Stopping  rule  for  solving  Bellman  equs 
under  existing  policies 

Else:  delta  =  delta 

End  If 


Next  k 

Loop  Until  delta  <  epsilon 

Application . StatusBar  =  False 
Turns  off  status  bar 
Close  #2 


End  Sub 
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i***************************************************  *  *  ****** 


Sub  policyimprove ( ) 

'Open  "c:\pol_imp"  For  Output  As  #1 

policystable  =  True 

For  k  =  1  To  numstates 


Checks  for 
better 
policies  for 
each  state 


Assume 
current 
policy  is 
best 

For  each 
state 

save  current 
policy  for 
state  k 


b  =  policystate (k) 
betterpolicy  =  policystate (k) 
previousvalue  =  -1000000000 
checkmod  =  k  Mod  numhours 

For  m  =  1  To  numactions 

storevalue  =0  Stores  value 

under  each 
policy 

policystate ( k)  =  a  (m) 
storeprob  =0 

If  k  <  numhours  And  checkmod  <>  0  Then  Top  row 

For  p  =  (k  +  1)  To  ((k  +  1)  +  numhours  *  (numcharge  -  1) ) 
Step  numhours 

Application .StatusBar  =  "policyiter  iter=  "  &  iter  &  "  k=  " 

&  k  &  "  action="  &  a (m)  &  "  p=  "  &  p 
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Call  calcprob 
Call  calccost 


storeprob  =  storeprob  +  myprob 

storevalue  =  storevalue  +  myprob  *  (cost  +  beta  * 
valuestate (p) ) 

If  (k  =  99)  Then  Print  #1,  "s  "  &  k  &  "  s'  "  &  p  &  "  prob  " 
&  myprob  &  "  cost  "  &  cost  &  "  loss  "  &  loss  &  "  expnl  "  & 
nl  &  "  wear  "  &  wear  &  "  value  "  &  storevalue 


Next  p 

Elself  checkmod  =  0  Then  Last 

Column 

For  p  =  1  To  (numhours  *  (numcharge  -  1)  +  1)  Step 
numhours 

Application . StatusBar  =  "policyiter  iter=  "  &  iter  &  "  k=  " 
&  k  &  "  action="  &  a (m)  &  "  p=  "  &  p 


Call  calcprob 
Call  calccost 


storeprob  =  storeprob  +  myprob 

storevalue  =  storevalue  +  myprob  *  (cost  +  beta  * 
valuestate (p) ) 

If  (k  =  99)  Then  Print  #1,  "s  "  &  k  &  "  s'  "  &  p  &  "prob  " 
&  myprob  &  "  cost  "  &  cost  &  "  loss  "  &  loss  &  "  expnl  "  & 
nl  &  "  wear  "  &  wear  &  "  value  "  &  storevalue 

Next  p 

Else  Not  top 

row  or 
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last 

column 

For  p  =  (timestate (k)  +2)  To  ( (numcharge  -  1)  * 

numhours  +  (timestate (k)  +  2))  Step  numhours 

Application. StatusBar  =  "policyiter  iter=  "  &  iter  &  "  k=  " 
&  k  &  "  action="  &  a(m)  &  "  p=  "  &  p 


Call  calcprob 
Call  calccost 


storeprob  =  storeprob  +  myprob 

storevalue  =  storevalue  +  myprob  *  (cost  +  beta  * 
valuestate (p) ) 


If  (k  =  99)  Then  Print  #1 ,  "s  "  &  k  &  "  s '  "  &  p  &  "  prob  " 
&  myprob  &  "  cost  "  &  cost  &  "  loss  "  &  loss  &  "  expnl  "  & 
nl  &  "  wear  "  &  wear  &  "  value  "  &  storevalue 

Next  p 

End  If 

If  (storevalue  >=  previous value)  Then 

previousvalue  =  storevalue 
betterpolicy  =  a (m) 

End  If 

Next  m 

policystate (k)  =  betterpolicy 
If  b  <>  policystate (k)  Then 
policystable  =  False 
End  If 
Next  k 
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Application. StatusBar  =  False 
Close  #1 

End  Sub 

i*********************************************************** 

Sub  calcprobO  Calculates 

transition 

probabilities 

If  (timestate (p)  =  timestate (k)  +  1)  Or 

(timestate (p)  =  timestate ( k)  -(numhours  -  1)) 

Then 

pt  =  1 

Else:  pt  =  0 

End  If 

If  socstate(p)  >  0  And  socstate(p)  <  100  Then 
Call  probstochl 

myprob  =  pt  *  ps 

Elself  socstate (p)  =  100  And  (socstate(k)  +  policystate (k)  - 

netload (timestate (p) ,  1))  >=  100 

Then 

Call  probstoch2  Ending  state  is 

at  top  of  state 
space 

myprob  =  pt  *  ps 

Elself  (socstate (p)  =  0)  And  (socstate (k)  +  policystate (k)  - 
netload (timestate (p) ,  countload (timestate (p) )) )  <=  0 


Ending  state  is 
in  middle  of 
state  space 


Then 
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Call  probstoch3  Ending  state  is 

at  bottom  of 
state  space 

myprob  =  pt  *  ps 

Else 

myprob  =0 
End  If 
End  Sub 

I*********************************************************** 

Sub  probstochlO  100>SOC(p)>0 

For  w  =  1  To  countload (timestate (p) ) 

If  (socstate(k)  +  policystate (k)  -  socstate(p))  <> 
netload (timestate (p) ,  w)  Then 

ps  =  0 

Else 

ps  =  probnetload (timestate (p) ,  w) 

GoTo  line20 

End  If 

Next  w 

line20 :  End  Sub 

I*********************************************************** 

Sub  probstoch2 ( )  SOC(p)=100 

tempprob  =0 

For  x  =  1  To  countload (timestate (p) )  Step  1 
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If  ( (socstate (k)  +  policystate (k)  -  netload (timestate (p) , 
x) )  >=  socstate (p) )  Then 

tempprob  =  tempprob  +  probnetload (timestate (p) ,  x) 
ps  =  tempprob 

Else 

GoTo  line40 
End  If 
Next  x 

line40 :  End  Sub 

r*  ★★★★★★★★*★**★*★★** **★★*****★★★*★★★★★★★★★★★★★*★**★***★***** 

Sub  probstoch3()  SOC(p)=0 

tempprob  =0 
tempnl  =0 

For  x  =  countload (timestate (p) )  To  1  Step  -1 

If  ( (socstate (k)  +  policystate (k)  -  netload (timestate (p) , 
x) )  <=  0 )  Then 

tempprob  =  tempprob  +  probnetload (timestate (p) ,  x) 
tempnl  =  tempnl  +  netload (timestate (p) ,  x)  * 
probnetload (timestate (p) ,  x) 

ps  =  tempprob 
nl  =  tempnl 

Else 

GoTo  line30 
End  If 
Next  x 

line30:  End  Sub 
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I  a********************************************************** 

Sub  calccost  ( )  Calculates  reward 

If  policystate ( k)  =  0  Then 

diesel  =0 
OM  =  0 

End  If 

If  policystate (k)  <>  0  Then 

diesel  =  -((0.08415  *  pr  +  0.246  *  policystate ( k) )  *  cf) 

Diesel  cost 

OM  =  -1 
End  If 

If  socstate(k)  >  socstate (p)  Then 

wear  =  -( (socstate (k)  -  socstate (p) )  *  battcost) 

Battery  wear  cost 

Else :  wear  =0 
End  If 

If  (socstate(p)  =  0)  And  (socstate(k)  +  policystate (k)  -  hi) 
<=  0  Then 

loss  =  -(nl  -  socstate(k)  -  policystate (k) )  * 
losspen  Loss  of  load  penalty 

Else :  loss  =0 
End  If 

cost  =  diesel  +  loss  +  OM  +  wear 


123 


Total  immediate  reward 

End  Sub 

!****★★ *************************** ******************* ★★*★★** 

Sub  load2()  Load 

resolution  2 
netloads 

For  i  =  1  To  24 

countload(i  -  1)  =  Worksheets ( "netloadprobs2" ). Cells (5, 

1  +  i) 

Next  i 

For  i  =  1  To  24 

For  j  =  1  To  countload(i  -  1) 
netload (i  -  1,  j )  = 

Worksheets ("netloadprobs2") .Cells (6  +  j,  1  +  i) 
probnetload (i  -  1,  j)  = 

Worksheets ("netloadprobs2") .Cells (68  +  j,  1  +  i) 

Next  j 
End  Sub 

i *********************************************************** 

Sub  load5()  Load 

resolution  5 
netloads 

For  i  =  1  To  24 

countload(i  -  1)  =  Worksheets ( "netloadprobs5" ). Cells (5, 
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1  +  i) 

Next  i 

For  i  =  1  To  24 

For  j  =  1  To  countload(i  -  1) 
netload (i  -  1,  j )  = 

Worksheets ("netloadprobs5") .Cells (6  +  j,  1  +  i) 
probnetload (i  -  1,  j)  = 

Worksheets ( "netloadprobs5" ). Cells (33  +  j,  1  +  i) 

Next  j 
Next  i 

End  Sub 

i *********************************************************** 

Sub  loadlOO  Load 

resolution 
10  netloads 


For  i  =  1  To  numhours 

countload(i  -  1)  =  Worksheets ("netloadprobslO") .Cells (5, 
1  +  i) 

Next  i 

For  i  =  1  To  numhours 

For  j  =  1  To  countload(i  -  1) 
netload (i  -  1,  j )  = 

Worksheets ( "netloadprobslO" ). Cells ( 6  +  j,  1  +  i) 
probnetload (i  -  1,  j)  = 

Worksheets ( "netloadprobslO" ). Cells (23  +  j,  1  +  i) 
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Next  j 
Next  i 


End  Sub 


Appendix  C 


OPTIMAL  DISPATCH  CHARTS 
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Figure  17.  Optimal  Strategy,  Net  Load  Profile  1,  Scenario 
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Figure  19.  Optimal  Strategy,  Net  Load  Profile  1,  Scenario 


Figure  20.  Optimal  Strategy,  Net  Load  Profile  1,  Scenario 
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Figure  33.  Optimal  Strategy,  Net  Load  Profile  2,  Scenario 


